Monday, 2019-11-25

*** rcernin has quit IRC00:17
*** rcernin has joined #openstack00:17
*** dsneddon has joined #openstack00:26
*** k_mouza has quit IRC00:53
*** skyraven has joined #openstack01:14
*** ircuser-1 has quit IRC01:17
*** skyraven has quit IRC01:19
*** decimuscorvinus has quit IRC01:27
*** dsneddon has quit IRC01:30
*** decimuscorvinus has joined #openstack01:30
*** decimuscorvinus has quit IRC01:30
*** lastmikoi has quit IRC01:32
*** decimuscorvinus has joined #openstack01:33
*** lastmikoi has joined #openstack01:38
*** nanzha has joined #openstack01:56
*** dsneddon has joined #openstack02:00
*** chenhaw has joined #openstack02:30
*** eb0t has quit IRC02:44
*** macz has joined #openstack02:54
*** macz has quit IRC02:59
*** skyraven has joined #openstack03:15
*** skyraven has quit IRC03:19
*** shibboleth has quit IRC03:32
*** cp has quit IRC03:35
*** vesper11 has quit IRC03:45
*** vesper11 has joined #openstack03:47
*** slaweq has joined #openstack03:58
*** davee__ has joined #openstack04:03
*** davee_ has quit IRC04:05
*** YusukeTatsumi has joined #openstack04:16
*** cp has joined #openstack04:20
*** YusukeTatsumi has quit IRC04:29
*** YusukeTatsumi has joined #openstack04:29
*** nanzha has quit IRC04:40
*** nanzha has joined #openstack04:41
*** YusukeTatsumi has quit IRC04:47
*** macz has joined #openstack04:54
*** macz has quit IRC04:58
*** nanzha has quit IRC05:00
*** nanzha has joined #openstack05:02
*** skyraven has joined #openstack05:16
*** msalo has joined #openstack05:16
*** soniya29 has joined #openstack05:18
*** skyraven has quit IRC05:20
*** msalo has quit IRC05:20
*** dsneddon has quit IRC05:32
*** dsneddon has joined #openstack05:34
*** dsneddon has quit IRC05:40
*** dsneddon has joined #openstack05:45
*** Warped has joined #openstack05:50
*** dsneddon has quit IRC05:50
*** jistr has quit IRC05:51
*** jistr has joined #openstack05:56
*** nanzha has quit IRC06:00
*** jistr has quit IRC06:06
*** jistr has joined #openstack06:06
*** ymasson has quit IRC06:09
*** nanzha has joined #openstack06:12
*** macz has joined #openstack06:14
*** macz has quit IRC06:19
*** rainmanjam has joined #openstack06:20
*** cp has quit IRC06:21
*** nanzha has quit IRC06:24
*** dsneddon has joined #openstack06:26
*** jtomasek has joined #openstack06:32
*** cp has joined #openstack06:34
*** nanzha has joined #openstack06:36
*** cp has quit IRC06:42
*** cp- has joined #openstack06:49
*** avivgta has joined #openstack06:56
*** bbowen has quit IRC06:57
*** skyraven has joined #openstack07:00
*** skyraven has quit IRC07:04
*** rcernin has quit IRC07:09
*** cah_link has joined #openstack07:11
*** CosmicSound4 has joined #openstack07:14
*** CosmicSound has quit IRC07:15
*** lpetrut has joined #openstack07:16
*** awalende has joined #openstack07:16
*** slaweq has quit IRC07:18
*** CosmicSound4 has quit IRC07:19
*** links has joined #openstack07:20
*** awalende has quit IRC07:21
*** nanzha has quit IRC07:22
*** CosmicSound has joined #openstack07:23
*** Vivas1984 has quit IRC07:25
*** jraju__ has joined #openstack07:27
*** links has quit IRC07:28
*** jangutter has joined #openstack07:33
*** CosmicSound is now known as cosmicsound07:35
*** dsneddon has quit IRC07:35
*** slaweq has joined #openstack07:35
*** nav2002 has joined #openstack07:38
*** akotobi has joined #openstack07:39
*** cosmicsound has quit IRC07:49
*** cosmicsound has joined #openstack07:54
*** dsneddon has joined #openstack08:00
*** nanzha has joined #openstack08:00
*** dsneddon has quit IRC08:05
*** dsneddon has joined #openstack08:12
*** suuuper has joined #openstack08:16
*** bengates has joined #openstack08:16
*** tesseract has joined #openstack08:16
*** rsimai has joined #openstack08:16
*** dsneddon has quit IRC08:16
*** maharg101 has quit IRC08:17
*** rsimai has quit IRC08:18
*** rsimai has joined #openstack08:19
*** maharg101 has joined #openstack08:19
*** fsimonce has joined #openstack08:21
*** dsneddon has joined #openstack08:22
*** dsneddon has quit IRC08:27
*** damien_r has joined #openstack08:29
*** macz has joined #openstack08:29
*** awalende has joined #openstack08:30
*** albert23 has quit IRC08:30
*** maharg101 has quit IRC08:33
*** rpittau|afk is now known as rpittau08:33
*** maharg101 has joined #openstack08:33
*** macz has quit IRC08:34
*** maharg101 has quit IRC08:34
*** awalende has quit IRC08:34
*** maharg101 has joined #openstack08:34
*** leanderthal has joined #openstack08:39
*** msalo has joined #openstack08:42
*** awalende has joined #openstack08:43
*** avivgta has quit IRC08:47
*** nanzha has quit IRC09:00
*** nanzha has joined #openstack09:01
*** skyraven has joined #openstack09:01
*** avivgt has joined #openstack09:04
*** skyraven has quit IRC09:05
*** Lucas_Gray has joined #openstack09:21
*** nav2002 has quit IRC09:21
*** avivgta has joined #openstack09:26
*** avivgta has quit IRC09:29
*** msalo has quit IRC09:29
*** Relih has joined #openstack09:30
*** msalo has joined #openstack09:30
*** avivgta has joined #openstack09:31
*** dsneddon has joined #openstack09:31
*** rgogunskiy has joined #openstack09:32
*** alexmcleod has joined #openstack09:32
*** cah_link1 has joined #openstack09:34
*** cah_link has quit IRC09:35
*** cah_link1 is now known as cah_link09:35
*** dsneddon has quit IRC09:37
*** k_mouza has joined #openstack09:42
*** k_mouza has quit IRC09:46
*** Zheyley has joined #openstack09:53
*** k_mouza has joined #openstack09:54
*** jistr is now known as jistr|afk09:58
*** pbandark has joined #openstack09:58
*** cah_link has quit IRC10:01
*** cah_link has joined #openstack10:01
*** takamatsu has quit IRC10:05
*** dsneddon has joined #openstack10:05
*** takamatsu has joined #openstack10:06
*** sshnaidm|off is now known as sshnaidm10:18
*** jistr|afk is now known as jistr10:22
*** msalo has quit IRC10:26
*** msalo_ has joined #openstack10:27
*** lastmikoi has quit IRC10:31
*** nanzha has quit IRC10:34
*** nanzha has joined #openstack10:38
*** suuuper has quit IRC10:42
*** akotobi has quit IRC10:46
*** random_yanek has quit IRC10:50
*** random_yanek has joined #openstack10:57
*** awalende_ has joined #openstack10:59
*** soniya29 has quit IRC10:59
*** skyraven has joined #openstack11:02
*** awalende has quit IRC11:02
*** awalende_ has quit IRC11:03
*** awalende has joined #openstack11:04
*** nanzha has quit IRC11:06
*** skyraven has quit IRC11:06
*** nanzha has joined #openstack11:06
*** brokencycle has joined #openstack11:07
*** chenhaw has quit IRC11:07
*** dsneddon has quit IRC11:10
*** awalende has quit IRC11:11
*** dsneddon has joined #openstack11:12
*** pcaruana has joined #openstack11:14
*** nav2002 has joined #openstack11:15
*** dsneddon has quit IRC11:17
*** Lucas_Gray has quit IRC11:19
*** fgonzales3246 has joined #openstack11:23
*** openstack has joined #openstack11:40
*** ChanServ sets mode: +o openstack11:40
*** openstack has joined #openstack11:52
*** ChanServ sets mode: +o openstack11:52
*** irclogbot_0 has joined #openstack11:54
*** arnoldoree has joined #openstack11:54
*** jtomasek has joined #openstack11:55
*** awalende has joined #openstack11:56
*** arnoldoree has quit IRC11:59
*** nanzha has quit IRC12:03
*** nanzha has joined #openstack12:04
*** eb0t has joined #openstack12:14
*** def_jam has quit IRC12:15
*** nanzha has quit IRC12:18
*** eblip has joined #openstack12:20
*** eb0t has quit IRC12:21
*** leanderthal has quit IRC12:26
*** leanderthal has joined #openstack12:26
*** Relih has joined #openstack12:28
*** nanzha has joined #openstack12:30
*** nav2002 has quit IRC12:33
*** nanzha has quit IRC12:36
*** nanzha has joined #openstack12:37
*** lastmikoi has joined #openstack12:44
*** Lucas_Gray has joined #openstack12:44
*** damien_r has quit IRC12:46
*** lpetrut has joined #openstack12:53
*** dsneddon has quit IRC12:56
*** jklare has quit IRC12:57
*** rgogunskiy has quit IRC12:57
*** rgogunskiy has joined #openstack12:57
*** drznpy has quit IRC12:58
*** drznpy has joined #openstack12:58
*** southquist has joined #openstack13:02
*** jklare has joined #openstack13:02
*** skyraven has joined #openstack13:03
*** dsneddon has joined #openstack13:04
*** servagem has joined #openstack13:04
*** skyraven has quit IRC13:07
*** dsneddon has quit IRC13:11
*** nanzha has quit IRC13:12
*** nanzha has joined #openstack13:18
*** zbr_ has joined #openstack13:18
*** msalo has joined #openstack13:19
*** zbr has quit IRC13:22
*** dsneddon has joined #openstack13:22
*** YokoBR has joined #openstack13:23
*** msalo_ has quit IRC13:23
YokoBRGood morning, everyone13:23
*** cgfbee has quit IRC13:23
*** nanzha has quit IRC13:24
YokoBRCinder is SO slow when using SSL. It's taking over 20 hourse to create an empty volume of 11gb.  I've found a problem with MySQL, 'Lost connection to MySQL server during query'13:26
*** dsneddon has quit IRC13:27
*** nanzha has joined #openstack13:30
*** dsneddon has joined #openstack13:32
*** damien_r has joined #openstack13:33
*** cgfbee has joined #openstack13:38
*** RickDeckard has joined #openstack13:41
*** dsneddon has quit IRC13:45
DHEYokoBR: well the mysql side is fixable... my.cnf [mysqld] wait_timeout=<bignumber>13:49
*** tonythomas has joined #openstack13:49
*** dsneddon has joined #openstack13:51
YokoBRDHE but that wouldn't make it be faster than 20 hours, right?13:51
*** KeithMnemonic has joined #openstack13:54
*** dsneddon has quit IRC13:55
DHEwell, no, but it will survive to the end of the operation...13:56
*** nanzha has quit IRC13:56
*** nanzha has joined #openstack13:59
*** dsneddon has joined #openstack14:01
YokoBRI see :(14:04
*** dsneddon has quit IRC14:06
*** dsneddon has joined #openstack14:12
*** dsneddon has quit IRC14:16
*** aconole has joined #openstack14:22
*** dsneddon has joined #openstack14:22
*** dsneddon has quit IRC14:26
*** Lucas_Gray has quit IRC14:27
*** jraju__ has quit IRC14:32
*** fraff has joined #openstack14:32
*** dsneddon has joined #openstack14:32
*** engine20191 has joined #openstack14:33
*** KeithMnemonic has quit IRC14:33
*** mcornea has joined #openstack14:34
*** soniya29 has quit IRC14:34
*** dsneddon has quit IRC14:36
*** dsneddon has joined #openstack14:39
*** dsneddon has quit IRC14:47
*** spiette has joined #openstack14:53
*** Zheyley has quit IRC14:55
*** leanderthal has quit IRC14:57
*** leanderthal has joined #openstack14:59
*** aedc_ has joined #openstack14:59
*** aedc has quit IRC15:02
*** skyraven has joined #openstack15:03
*** drewlander has joined #openstack15:04
*** aedc_ has quit IRC15:04
*** aedc has joined #openstack15:05
*** skyraven has quit IRC15:08
*** aedc has quit IRC15:10
*** suuuper has joined #openstack15:11
*** awalende has quit IRC15:11
*** dsneddon has joined #openstack15:13
*** awalende has joined #openstack15:16
*** awalende has quit IRC15:17
*** dsneddon has quit IRC15:18
*** rgogunskiy has quit IRC15:21
*** dsneddon has joined #openstack15:22
*** dsneddon has quit IRC15:26
*** aedc has joined #openstack15:29
*** dsneddon has joined #openstack15:32
*** dsneddon has quit IRC15:37
*** dsneddon has joined #openstack15:43
*** Goneri has joined #openstack15:47
*** glguy has joined #openstack15:47
*** msalo has quit IRC15:47
*** dsneddon has quit IRC15:48
*** pbandark has quit IRC15:50
*** dsneddon has joined #openstack15:53
*** macz has joined #openstack15:56
*** devfaz has quit IRC15:56
*** macz has quit IRC15:56
*** devfaz has joined #openstack15:57
*** msalo has joined #openstack16:04
*** glguy has left #openstack16:06
*** openstackstatus has joined #openstack16:14
*** ChanServ sets mode: +v openstackstatus16:14
*** rgogunskiy has joined #openstack16:17
*** nanzha has quit IRC16:18
*** Warped has quit IRC16:20
*** southquist has quit IRC16:21
*** rgogunskiy has quit IRC16:22
*** suuuper has quit IRC16:25
*** idlemind has joined #openstack16:25
*** fandi has joined #openstack16:30
*** RickDeckard has quit IRC16:31
*** schwicht has joined #openstack16:34
*** ircuser-1 has joined #openstack16:37
*** YokoBR has quit IRC16:37
*** rpittau is now known as rpittau|afk16:40
*** marst has joined #openstack16:42
*** Relih has quit IRC16:43
*** fandi_ has joined #openstack16:44
*** ymasson has joined #openstack16:46
*** fandi has quit IRC16:47
*** lpetrut has quit IRC16:51
*** RickDeckard has joined #openstack16:59
*** msalo_ has joined #openstack17:03
*** dsneddon has quit IRC17:03
*** skyraven has joined #openstack17:04
*** msalo has quit IRC17:06
*** avivgta has quit IRC17:07
*** skyraven has quit IRC17:09
*** dsneddon has joined #openstack17:09
*** fgonzales3246 has quit IRC17:12
*** awalende has joined #openstack17:12
*** shibboleth has joined #openstack17:14
*** dsneddon has quit IRC17:14
*** awalende has quit IRC17:17
*** msalo has joined #openstack17:17
*** igordc has joined #openstack17:18
*** damien_r has quit IRC17:19
*** dsneddon has joined #openstack17:19
*** msalo_ has quit IRC17:20
*** schwicht has quit IRC17:20
*** jathan has joined #openstack17:21
*** takamatsu has quit IRC17:27
*** bengates has quit IRC17:29
*** FlorianFa has quit IRC17:29
*** dsneddon has quit IRC17:30
*** takamatsu has joined #openstack17:32
*** schwicht has joined #openstack17:35
*** dudek has joined #openstack17:39
*** k_mouza has quit IRC17:47
gregworkdoes anyone here use collectd+gnocchi to collect metrics/meaures from their cloud?  i have a very small cloud (6 nodes) and enabling telemetry has created a persistent 1000-1400 IOPs increase on my metrics pool in ceph. all tiny reads/writes (4k) but its super noticable17:54
gregworkim curious if this is expected17:54
*** avivgt has quit IRC17:54
*** skyraven has joined #openstack17:55
*** aedc_ has joined #openstack17:55
*** tesseract has quit IRC17:57
*** RickDeckard has quit IRC17:58
*** aedc has quit IRC17:58
*** aedc_ has quit IRC18:00
*** macz has joined #openstack18:03
*** RickDeckard has joined #openstack18:03
*** cah_link has quit IRC18:09
*** albert23 has joined #openstack18:14
*** arnoldoree has joined #openstack18:17
*** jtomasek has quit IRC18:19
*** brokencycle has quit IRC18:26
*** k_mouza has joined #openstack18:28
*** dsneddon has joined #openstack18:29
*** k_mouza has quit IRC18:33
*** cah_link has joined #openstack18:34
*** igordc has quit IRC18:39
*** dsneddon has quit IRC18:44
*** dsneddon has joined #openstack18:49
*** msalo has quit IRC18:56
*** msalo has joined #openstack18:57
*** imega has joined #openstack18:59
*** cah_link has quit IRC18:59
*** Warped has joined #openstack19:02
*** engine20191 has quit IRC19:03
*** alexmcleod has quit IRC19:08
*** shibboleth has quit IRC19:09
*** msalo_ has joined #openstack19:13
*** deep13 has quit IRC19:14
*** k_mouza has joined #openstack19:15
*** deep13 has joined #openstack19:16
*** k_mouza has quit IRC19:16
*** k_mouza has joined #openstack19:17
*** RickDeckard has quit IRC19:17
*** msalo has quit IRC19:18
*** RickDeckard has joined #openstack19:18
*** rgogunskiy has joined #openstack19:18
*** rgogunskiy has quit IRC19:20
*** macz has quit IRC19:21
*** k_mouza has quit IRC19:32
*** Goneri has quit IRC19:39
*** boazel has quit IRC19:43
*** fandi_ has quit IRC19:44
*** boazel has joined #openstack19:46
*** igordc has joined #openstack19:55
*** servagem has quit IRC20:06
*** Goneri has joined #openstack20:10
*** nav2002 has joined #openstack20:10
*** thobias_trevisan has joined #openstack20:18
*** thobias_trevisan has quit IRC20:19
*** servagem has joined #openstack20:23
*** RickDeckard has quit IRC20:24
*** RickDeckard has joined #openstack20:25
*** macz has joined #openstack20:46
*** igordc has quit IRC20:47
*** shibboleth has joined #openstack20:50
*** igordc has joined #openstack20:53
*** malevolent has joined #openstack21:11
*** schwicht has quit IRC21:24
*** k_mouza has joined #openstack21:24
*** k_mouza has quit IRC21:28
*** rcernin has joined #openstack21:31
*** biyiklioglu has joined #openstack21:32
*** brokencycle has joined #openstack21:34
*** k_mouza has joined #openstack21:34
*** pcaruana has quit IRC21:38
*** biyiklioglu has quit IRC21:38
*** ntpttr has joined #openstack21:45
*** k_mouza has quit IRC21:48
*** msalo has joined #openstack22:03
*** slaweq has quit IRC22:06
*** freefood_ has quit IRC22:07
*** msalo_ has quit IRC22:07
*** arnoldoree has quit IRC22:07
*** RickDeckard has quit IRC22:18
*** Esras-Work has joined #openstack22:20
Esras-WorkHi there. I'm trying to debug an issue that I've got with the metadata service on a Train installation, and I'm not sure where to go next. Is this the right channel or should I go somewhere else?22:22
*** KeithMnemonic has joined #openstack22:41
ozzzoThis is technically the right channel, but good luck getting help here22:41
ozzzoI usually ask here, and nobody answers, then I try various other openstack-* channels, then I send an email to the list22:41
ozzzobut go ahead and ask; maybe I will know the answer22:42
Esras-WorkHa, thanks. So the issue I'm having is that the metadata service is returning a 500 from the VM whenever the VM does a call to 169.254.169.254/*. Calling the base address without anything else returns the list of appropriate protocols, but calling any of them gets a "Remote metadata server experienced an internal server error." Digging into it, it looks like the metadata-agent passes things along22:48
Esras-Workappropriately to the nova-api service, but it generates an error for "Failed to get metadata for instance id: X: NotFound." I'm using only one Region and theoretically everything should be under the Default namespace, and I'm using the Puppet modules to configure things. I've verified that both the metadata-agent and the nova.conf file are using the same shared_secret. I'm trying to look into the DB22:48
*** msalo has quit IRC22:48
Esras-Worknow and see if it's just not there for some reason.22:48
*** slaweq has joined #openstack22:49
*** aconole has quit IRC22:54
*** mcornea has quit IRC23:07
*** tonythomas has quit IRC23:10
*** slaweq has quit IRC23:10
ozzzoEsras-Work: I've seen that before; it usually indicates that the VM is not able to contact the DHCP server23:13
ozzzoit can't get an IP, so then it can't do the metadata call23:13
ozzzoif you look at the vm console, does it have a correct hostname, or does it say "localhost" ?23:13
ozzzooh wait, 500? I think I was getting a different error23:14
Esras-Work@ozzzo: Yeah, 500 from the VM side, but a 404 from "nova.api.metdata.handler" VM does have an IP address23:20
ozzzoThat's not the issue I saw then; it looks like you may have an actual metadata server issue23:20
*** fsimonce has quit IRC23:21
Esras-WorkHa! I've been spending a few days trying to scour any sort of ask.openstack questions and I keep finding other issues, but not this one. I even tried a complete reinstall from scratch, but I still want to assume that this is a missconfiguration on my part.23:21
ozzzoI think I would try #openstack-nova23:21
ozzzoor the list if you don't get an answer there23:21
Esras-WorkAlright, thanks.23:23
Esras-WorkOn an interesting note, I just found in my keystone_wsgi_access log that something is trying to POST /auth/tokens. That's not right...23:23
*** dudek has quit IRC23:25
*** imega has quit IRC23:27
ozzzoThat does seem incorrect23:27
ozzzoThe token examples are all GET23:27
ozzzowe are on rocky so I use this for reference: https://docs.openstack.org/keystone/rocky/api_curl_examples.html23:28
ozzzoDoes anyone know how to get out of the "lb-haproxy-clone is not running anywhere and so cannot be restarted" failure without rebooting?23:30
ozzzohttp://dpaste.com/33T3T8523:30
Esras-WorkOh my gosh, that was it. So by default, due to this bug: https://bugs.launchpad.net/tripleo/+bug/1580076, the "neutron_auth_type" parameter in manifests/network/neutron.pp is set to "v3password" instead of "password," which means that if you use the same unversioned endpoint as is expected in almost every other module, you end up getting failures and thus the VM fails to get metadata because it 404s23:38
openstackLaunchpad bug 1580076 in tripleo "Upgrades job failing pingtest with "Message: No valid host was found."" [Critical,Fix released] - Assigned to Dan Prince (dan-prince)23:38
Esras-Workagainst _keystone_, not against anything else.23:38
*** jathan has quit IRC23:38
*** schwicht has joined #openstack23:45
*** cosmicsound has quit IRC23:45
*** cosmicsound has joined #openstack23:45
*** k_mouza has joined #openstack23:49
*** nav2002 has quit IRC23:49
Esras-WorkYup. It works now. I have spent... many hours debugging that.23:49
ozzzonice23:49
Esras-WorkAlright, well, onto the next step. Any chance you might be able to point me towards a doc for integrating Puppet with OpenStack VMs, so that I can pre-puppetize a VM as it's deployed?23:50
ozzzoWe're using Ansible, but there is a Puppet guide:23:51
ozzzohttps://docs.openstack.org/puppet-openstack-guide/latest/23:51
Esras-WorkYup, already using that for deployment.23:51
*** k_mouza has quit IRC23:53
Esras-WorkThanks, @ozzzo for your help23:54
ozzzoyw23:54
*** schwicht has quit IRC23:54
*** jklare has quit IRC23:56
*** macz has quit IRC23:57
*** macz has joined #openstack23:58
*** jklare has joined #openstack23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!