| notmyname | clarkb: perhaps, except https://github.com/openstack/swift/blob/master/test/functional/tests.py#L1207 | 00:00 |
|---|---|---|
| clarkb | notmyname: I wonder if we are using a different value then or possibly just for the functional tests | 00:01 |
| clarkb | also the functional tests run serially so it doesn't completely rule out the concurrent request theory. /me looks at the functional test config | 00:01 |
| *** loq_mac has joined #openstack-infra | 00:02 | |
| notmyname | clarkb: do you have a swift/test.conf https://github.com/openstack/swift/blob/master/test/sample.conf | 00:02 |
| fungi | ahh, right, the bumps we've been seeing so far are to SWIFT_LOOPBACK_DISK_SIZE | 00:02 |
| clarkb | notmyname: that appears to be the config used by the functional tests in the gate and it uses the default max file size | 00:03 |
| *** salv-orlando has joined #openstack-infra | 00:04 | |
| clarkb | is it possible that the loopback can eb smaller than the largest object if there is only one object? | 00:04 |
| clarkb | or maybe swift isn't running on the loopback? | 00:05 |
| notmyname | clarkb: that would depend on the devices setting and where the loopback is mounted | 00:06 |
| salv-orlando | I have a query regarding the large-ops job for neutron. Can I get a few minutes from anyone of the infra people? | 00:06 |
| clarkb | salv-orlando: you probably want jog0 who was around just a few minutes ago, I can help if he has disappeared | 00:07 |
| salv-orlando | thanks clarkb. I will post my request and then go afk for a coffee. Hopefully jog0 will read it | 00:09 |
| jog0 | salv-orlando: pong | 00:10 |
| salv-orlando | We have looked into several failing jobs - there are no deadlocks or timeouts within neutron. All the 150 ports are always correctly created. We initially expected allocate_for_instance to block | 00:10 |
| salv-orlando | hei jog0 | 00:10 |
| salv-orlando | Since the largeops driver uses the fake dirt driver it does not make a lot of sense to query for errors in the L2 agent | 00:10 |
| *** julim has quit IRC | 00:11 | |
| salv-orlando | I am starting to be convinced that there is actually nothing really wrong with the sequence of operations. However, a few db changes introduced around the critical date (Oct 31st), add SELECT FOR UPDATE statements | 00:12 |
| jog0 | salv-orlando: so it may be things are just too slow | 00:12 |
| salv-orlando | this might mean less concurrency, and more time to complete request | 00:12 |
| salv-orlando | exactly | 00:12 |
| jog0 | so a bit of background: | 00:12 |
| salv-orlando | I'd propose to try and change BUILD_TIMEOUT to 300 in devstack-gate only for neutron | 00:12 |
| jog0 | -1 | 00:13 |
| jog0 | although that is a good test if we don't merge it | 00:13 |
| *** loq_mac has quit IRC | 00:13 | |
| jog0 | salv-orlando: large-ops is the regression test to the issues nova-networking had around rootwrap and iptables | 00:13 |
| jog0 | where it took forever to boot even 50 instances (RPC calls would timeout after a minute so things would fail) | 00:14 |
| *** pcrews has quit IRC | 00:16 | |
| clarkb | notmyname: it appears to be truncating the SWIFT_DISK_IMAGE to LOOPBACK_SIZE, creating an xfs fs on SWIFT_DISK_IMAGE then mounting that with -o loop on SWIFT_DATA_DIR/drives/sdb1 | 00:16 |
| jog0 | salv-orlando: if neutron is this slow IMHO its broken | 00:16 |
| clarkb | notmyname: I assume that swift knows to look in SWIFT_DATA_DIR/drives for its drives? | 00:17 |
| lifeless | mordred: https://review.openstack.org/#/c/54817/1 needs unabandoning | 00:17 |
| salv-orlando | jog0: agreed. I wanted just to validate the hypothesis. But for that I should be able to use devstack-gate and largeops locally. | 00:18 |
| notmyname | clarkb: yes | 00:18 |
| *** matsuhashi has joined #openstack-infra | 00:18 | |
| salv-orlando | And if largeops should never fail locally, that would be a further validation of this hypothesis | 00:18 |
| jog0 | salv-orlando: so there is a gotcha with doing it locally | 00:18 |
| notmyname | clarkb: and so the question remains, why is it failing now and why is this a new failure? | 00:19 |
| jog0 | which is this is perforamnce dependant so you may get different results | 00:19 |
| salv-orlando | indeed | 00:19 |
| jog0 | salv-orlando: so just make an upstream patch to confirm your theory then mark as do not merge | 00:19 |
| salv-orlando | that's why if it turns out to succeed always because my machines are more performant, this would mean that the issue is that neutron has slowed down | 00:19 |
| salv-orlando | jog0: I can do a devstack-gate patch, but would then gerrit run the largeops job for neutron? | 00:20 |
| jog0 | salv-orlando: ohh good point yes | 00:20 |
| jog0 | salv-orlando: not sure, clarkb would know. if not there are other places you can wire in like tempest to test | 00:20 |
| salv-orlando | jog0: we can turn the problem the other way. | 00:20 |
| salv-orlando | I will revert the patches which introduced SELECT FOR UPDATE - this should result in failures in the 'standard' jobs, but largeops should be back to green | 00:21 |
| jog0 | salv-orlando: cool, i like it | 00:21 |
| jog0 | good test | 00:21 |
| salv-orlando | ok, I'll do that then. | 00:21 |
| clarkb | notmyname: the functional tests still pass, and are a place I would expect to fail since they test the boundaries of max_file_size | 00:23 |
| notmyname | clarkb: sorry for all the delays in responding. I was working on something for a meeting. and that meeting just got canceled | 00:23 |
| notmyname | clarkb: exactly | 00:23 |
| clarkb | notmyname: no problem, I have been digging through devstack to understand the setup, just about ready to look at actual tests now | 00:24 |
| anteaya | salv-orlando jog0 awesome | 00:24 |
| *** milki has quit IRC | 00:25 | |
| *** milki has joined #openstack-infra | 00:26 | |
| clarkb | nose claims the test runs and passes on our devstack slaves | 00:26 |
| *** dkliban has quit IRC | 00:29 | |
| *** paul--_ has joined #openstack-infra | 00:29 | |
| *** jcooley_ has joined #openstack-infra | 00:29 | |
| *** markwash has quit IRC | 00:29 | |
| *** jcooley_ has quit IRC | 00:32 | |
| *** hogepodge has quit IRC | 00:33 | |
| clarkb | http://logs.openstack.org/57/54857/6/check/check-swift-devstack-vm-functional/e29a7a7/logs/devstacklog.txt.gz#_2013-11-11_23_48_14 | 00:34 |
| clarkb | we are setting the image size to 1G in the functional tests because tempest is not enabled | 00:34 |
| *** jcooley_ has joined #openstack-infra | 00:34 | |
| notmyname | ah ok. which is why it works | 00:34 |
| notmyname | and tempest isn't running the functional tests | 00:35 |
| clarkb | you need to scroll down a little ways below that anchor point | 00:35 |
| notmyname | I'll take your word for it :-) | 00:35 |
| clarkb | notmyname: why would it work there through, max file size is 5G | 00:35 |
| clarkb | 1G << 5G | 00:35 |
| clarkb | when tempest runs the image size is 4G still less than 5G | 00:36 |
| notmyname | clarkb: I don't think tempest does a file size check | 00:36 |
| clarkb | notmyname: oh I see so tempest doesn't know better until something fails | 00:36 |
| clarkb | notmyname: shouldn't the functional tests fail then as they do a file size check? | 00:37 |
| notmyname | except the test config is set to the disk size. so even though the server has it at 5GB, it still responds with the "correct" (to the test) response because the fallocate doesn't work | 00:37 |
| *** moted has quit IRC | 00:38 | |
| notmyname | so it passes, but it's an accident | 00:38 |
| *** sarob has joined #openstack-infra | 00:38 | |
| clarkb | gotcha, so we should set the image size to ~10G to double the default 5G size? | 00:39 |
| clarkb | I don't want to reduce the file size because testing defaults is valuable | 00:39 |
| notmyname | clarkb: if you have the space, that would be good. of course, there's also great value in testing non-defaults too ;-) | 00:40 |
| clarkb | fungi: did you follow any of that? have an opinion if you did? | 00:41 |
| notmyname | clarkb: of course, devstack itsef is a pretty weird test setup. "testing a massive distributed storage system? let's provision one loopback device with one replica on one vm!" ;-) | 00:42 |
| *** gyee has quit IRC | 00:43 | |
| *** sarob has quit IRC | 00:43 | |
| clarkb | notmyname: fair :) | 00:43 |
| notmyname | clarkb: I'm working on fixing that, kinda. I'm trying to get a cluster of real hardware set up dedicated to community swift testing. (thus my question earlier about failure notifications for "nightly" tests) | 00:44 |
| *** mriedem has joined #openstack-infra | 00:45 | |
| lifeless | notmyname: how much hardware do you need? | 00:46 |
| lifeless | notmyname: could you be another test app on the TripleO cloud ? | 00:46 |
| clarkb | notmyname: oh so ya, lifeless is trying to do the tripleo cloud stuff, that may be one way of getting enough hardware to actually report on every patch/change | 00:47 |
| notmyname | lifeless: hmm...interesting. | 00:47 |
| clarkb | become part of the larger distributed machine | 00:47 |
| notmyname | lifeless: honestly I don't want to start that conversation today. can I talk to you about it tomorrow? (I wish I would have known so we could have talked in HK) | 00:48 |
| lifeless | notmyname: so yeah we're looking at bare metal gating and multi-node tests etc | 00:48 |
| lifeless | notmyname: anytime is good | 00:48 |
| lifeless | notmyname: if all the vendors who verbally committed stuff come through we'll have about 200 machines spread over 8-10 regions | 00:48 |
| lifeless | notmyname: of many different sizes | 00:48 |
| notmyname | lifeless: nice | 00:49 |
| *** dkranz has joined #openstack-infra | 00:51 | |
| lifeless | notmyname: my estimate is that it will take 80 machines to handle bare metal gate load for one hypervisor configuration (including Ironic narrow-focused tests) | 00:51 |
| lifeless | notmyname: anyhow, yes - tomorrow. | 00:51 |
| notmyname | lifeless: so you're saying I can use 120 machines? :-) | 00:51 |
| lifeless | notmyname: no, because we also need emulated baremetal for tripleo + ironic check jobs | 00:52 |
| notmyname | lol | 00:52 |
| lifeless | notmyname: and at least one chunk of those machines is from a hypervisor vendor wanting gate-level testing who can't run virt | 00:53 |
| lifeless | notmyname: but! swift was always part of the tripleo story we're aiming at, so the load we're anticipating already includes basic swift. Your advice on taking it from devstack style swift to production representative stuff is going to be essential | 00:53 |
| notmyname | lifeless: cool. | 00:54 |
| notmyname | lifeless: are you back in NZ timezone? | 00:54 |
| lifeless | I am | 00:54 |
| lifeless | for at least 3 days | 00:54 |
| notmyname | lifeless: kk. I'll ping you later about it | 00:54 |
| openstackgerrit | Russell Bryant proposed a change to openstack-dev/cookiecutter: Add newline to openstack-common.conf https://review.openstack.org/55120 | 00:56 |
| *** yamahata has joined #openstack-infra | 00:56 | |
| openstackgerrit | Russell Bryant proposed a change to openstack-dev/cookiecutter: Add newline to openstack-common.conf https://review.openstack.org/55120 | 00:58 |
| jerryz | clarkb: does zuul refs get replicated to git.o.o ? if so , i could not find the code doing that. just confirm with you about it. Thanks | 00:58 |
| *** dims has quit IRC | 01:00 | |
| clarkb | jerryz: we did that in the past but now don't replicate them back to gerrit or git.o.o as they were too noisy | 01:00 |
| clarkb | they are now just hosted from zuul directly | 01:00 |
| jerryz | clarkb: i see. thank you | 01:03 |
| *** adalbas has quit IRC | 01:04 | |
| *** wenlock has quit IRC | 01:06 | |
| *** nosnos has joined #openstack-infra | 01:12 | |
| *** mikal has quit IRC | 01:14 | |
| *** dims has joined #openstack-infra | 01:15 | |
| *** moted has joined #openstack-infra | 01:15 | |
| *** mikal has joined #openstack-infra | 01:16 | |
| *** bknudson has joined #openstack-infra | 01:17 | |
| bknudson | I think keystone is f'd somehow. | 01:17 |
| bknudson | when I update a new patch set Jenkins -1 with can't merge... | 01:17 |
| bknudson | https://review.openstack.org/#/c/54647/ | 01:17 |
| *** ArxCruz has joined #openstack-infra | 01:18 | |
| fungi | lifeless: i've unabandoned 54817 in case you were considering updating it | 01:19 |
| lifeless | fungi: thanks; there are some N stacked on top of it ;) | 01:20 |
| lifeless | fungi: if you can answer my question about it, that would be great | 01:20 |
| clarkb | bknudson: can confirm that arbitrary commit off of keystone master is unhappy | 01:21 |
| clarkb | fungi: https://review.openstack.org/#/c/53222/ I think zuul is derped | 01:22 |
| *** jcooley_ has quit IRC | 01:22 | |
| *** jhesketh__ has joined #openstack-infra | 01:22 | |
| fungi | clarkb: back now. followed on the max_file_size, and i'd say set it to a reasonable quantity for our current slaves (which is way huge--i think we've been incrementally bumping disk-related values based on the old 2gb flavor block device size, which is hella-small, though keeping it reasonable for people running this on their laptops is also meritorious i think) | 01:24 |
| clarkb | ahahahahaha | 01:26 |
| zaro | fungi: are the titles here generated or manuanlly set? http://docs.openstack.org/infra/publications/ | 01:26 |
| clarkb | too many open files | 01:26 |
| * jog0 just wants the gate to to be stable | 01:26 | |
| clarkb | fungi: I think you dealt with this previously | 01:26 |
| clarkb | fungi: should I just restart zuul? | 01:26 |
| clarkb | maybe capture an lsof first? | 01:26 |
| fungi | clarkb: too many open files on zuul? get a thread dump first | 01:26 |
| zaro | one of the links points to my slides, but name of link is incorrect. | 01:27 |
| fungi | zaro: generated from... somewhere. checking | 01:27 |
| clarkb | fungi: yes | 01:27 |
| fungi | and then, yes, restarting zuul should clear them | 01:27 |
| fungi | or at least it did for me previously | 01:27 |
| clarkb | ok I will do lsof and thread dump then restart zuul | 01:28 |
| fungi | also if you can, note how long zuul's been running (though i suspect this issue is more proportional to the number of changes it's tested rather than the duration it's been running) | 01:28 |
| clarkb | fungi: since october 25th, there are several zuul-* fiels in my homedir on the zuul server that capture lsof, ps, and the tail of the log (including thread stacks) | 01:31 |
| clarkb | fyi on where they are located | 01:32 |
| clarkb | restarting zuul now | 01:32 |
| fungi | zaro: title is pulled from README.rst in your branch | 01:32 |
| fungi | looks like it hasn't been updated | 01:33 |
| fungi | the hope is that it can contain a reusable abstract we can submit to a typical conference cfp, and also talking points for the slides | 01:33 |
| fungi | so, in theory, we can repeat the talk at other conferences | 01:34 |
| clarkb | fungi: I think it is better now but the wifi on my laptop is derping | 01:35 |
| clarkb | fungi: can you get to zuul.o.o via ssh? is it happy? | 01:35 |
| clarkb | bknudson: I think we have fixed (at least temporarily) the problem | 01:35 |
| fungi | i'm watching the status page for a keystone pep8 job which is about to finish | 01:35 |
| zaro | ahh. cool thanks. | 01:35 |
| zaro | fungi: ^ | 01:35 |
| *** michchap has quit IRC | 01:35 | |
| fungi | zaro: yw | 01:35 |
| *** mrodden has quit IRC | 01:36 | |
| clarkb | zaro: did you see what happened with jenkins-ci github repos? | 01:36 |
| *** michchap has joined #openstack-infra | 01:36 | |
| clarkb | you should double check that gearman-plugin survived ok when you have time | 01:36 |
| bknudson | clarkb: thanks! | 01:36 |
| clarkb | fungi: the lsof capture looks like tons of fds into git repos | 01:36 |
| clarkb | fungi: I bet merger.py can create Repo() objects on demand and GC them when done to avoid that | 01:37 |
| clarkb | but would need some time reading code to double check that doing so isn't insane | 01:37 |
| fungi | got a linecount? sudo lsof|wc -l sez 1882 right now, so seems low/reasonable | 01:37 |
| bknudson | do I recheck no bug? | 01:37 |
| clarkb | bknudson: yes | 01:38 |
| fungi | bknudson: click the review button on your change | 01:38 |
| fungi | oh, you said "do i" not "how do i" | 01:38 |
| fungi | ignore me ;) | 01:38 |
| clarkb | fungi: not until I can convince my laptop to connect me to zuul.o.o. you can look at the file in my homedir | 01:38 |
| bknudson | wasn't sure if I had to rebase. | 01:38 |
| clarkb | bknudson: you shouldn't need to rebase | 01:38 |
| clarkb | fungi: I did lsof | grep zuul > zuul-lsof.txt or some such and the wc on that file was ~2200 | 01:38 |
| fungi | yeah, just leave a review comment which says that three-word phrase and nothing else | 01:38 |
| *** erfanian has joined #openstack-infra | 01:39 | |
| fungi | fungi@zuul:~$ sudo lsof|grep -c zuul | 01:39 |
| fungi | 592 | 01:39 |
| fungi | seems lower | 01:39 |
| clarkb | * wc -l was 2200 but I think you grokked | 01:39 |
| fungi | yup | 01:40 |
| fungi | also, jobs succeeding | 01:40 |
| *** fifieldt has quit IRC | 01:40 | |
| *** rwsu is now known as rwsu-away | 01:40 | |
| fungi | which tells me it's successfully creating new git refs | 01:41 |
| * fungi transitions to less efficient typing apparatus in a more residential part of his domicile | 01:41 | |
| clarkb | fungi: cool thanks for checking | 01:42 |
| clarkb | fungi: I looked at the lsof capture against and it looks like 2004 fd were in /var/lib/zuul/git | 01:51 |
| fungi | clarkb: sounds just like last time | 01:51 |
| fungi | the limit is around 2048 | 01:51 |
| fungi | suspicion is that the git.Repo instances are being held open after they're no longer in use, and need a del() or similar aggressive gc added | 01:52 |
| clarkb | fungi: they are being assigned to Repo.repo which is keeping them around I think | 01:53 |
| fungi | seems to be a fairly recent behavior, so possibly introduced with the nnfi changes or after | 01:53 |
| clarkb | fungi: I believe that Repo can let them go since it needs to recreate the objects in most cases anyways | 01:53 |
| fungi | ahh, yeah. i haven't had time to dig into the source for answers, but that sounds likely | 01:54 |
| *** erfanian has quit IRC | 01:55 | |
| clarkb | I will whip something up really quickly to test that theory and remind us that the work needs to be done | 01:55 |
| *** ljjjustin has joined #openstack-infra | 01:56 | |
| *** yaguang has joined #openstack-infra | 01:56 | |
| *** dims has quit IRC | 01:57 | |
| *** dims has joined #openstack-infra | 01:57 | |
| *** mattyb has quit IRC | 01:58 | |
| *** nosnos_ has joined #openstack-infra | 02:01 | |
| *** jerryz has quit IRC | 02:04 | |
| *** nosnos has quit IRC | 02:04 | |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/zuul: Make merger.Repo more context manager like. https://review.openstack.org/55943 | 02:05 |
| clarkb | fungi: I will WIP that because I know it is incomplete. But suggestions on making it cleaner would be nice | 02:05 |
| clarkb | I am not particularly happy with that approach. I think making merger.Repo implicitly context manage its internal operations may be better. or maybe just make the whole thing a context manager | 02:10 |
| *** jerryz has joined #openstack-infra | 02:15 | |
| *** bingbu has joined #openstack-infra | 02:15 | |
| clarkb | comments more than welcome, and now I need to find dinner | 02:15 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Better end of console log indexing detection. https://review.openstack.org/53487 | 02:19 |
| *** marun has joined #openstack-infra | 02:20 | |
| *** ljjjusti1 has joined #openstack-infra | 02:20 | |
| *** mrodden has joined #openstack-infra | 02:21 | |
| *** marun has quit IRC | 02:21 | |
| *** marun has joined #openstack-infra | 02:21 | |
| *** mrodden1 has joined #openstack-infra | 02:23 | |
| *** ljjjustin has quit IRC | 02:23 | |
| *** mrodden has quit IRC | 02:25 | |
| *** guohliu has joined #openstack-infra | 02:29 | |
| *** nosnos_ has quit IRC | 02:30 | |
| *** nosnos has joined #openstack-infra | 02:31 | |
| *** jcooley_ has joined #openstack-infra | 02:42 | |
| *** matsuhashi has quit IRC | 02:47 | |
| *** marun has quit IRC | 02:50 | |
| *** mriedem has quit IRC | 02:50 | |
| openstackgerrit | DennyZhang proposed a change to openstack-infra/config: remove code duplication in run-*.sh https://review.openstack.org/54646 | 02:51 |
| *** melwitt has quit IRC | 03:02 | |
| *** ian_ott has quit IRC | 03:08 | |
| *** guohliu has quit IRC | 03:15 | |
| *** jerryz has quit IRC | 03:15 | |
| *** guohliu has joined #openstack-infra | 03:29 | |
| *** yamahata has quit IRC | 03:32 | |
| *** matsuhashi has joined #openstack-infra | 03:35 | |
| *** jcooley_ has quit IRC | 03:38 | |
| *** jcooley_ has joined #openstack-infra | 03:39 | |
| *** jcooley_ has quit IRC | 03:43 | |
| *** UtahDave has quit IRC | 03:50 | |
| *** senk has joined #openstack-infra | 03:51 | |
| *** oubiwann has quit IRC | 03:56 | |
| *** CaptTofu has quit IRC | 03:59 | |
| *** CaptTofu has joined #openstack-infra | 04:00 | |
| *** markwash has joined #openstack-infra | 04:00 | |
| *** ArxCruz has quit IRC | 04:01 | |
| *** dims has quit IRC | 04:03 | |
| *** CaptTofu has quit IRC | 04:05 | |
| *** chandankumar has joined #openstack-infra | 04:08 | |
| *** talluri has joined #openstack-infra | 04:10 | |
| *** talluri_ has quit IRC | 04:11 | |
| openstackgerrit | Dan Stangel proposed a change to openstack-infra/gitdm: Fix broken email mapping and add missing domains https://review.openstack.org/53419 | 04:16 |
| *** paul--_ has quit IRC | 04:17 | |
| *** markwash has quit IRC | 04:20 | |
| *** talluri_ has joined #openstack-infra | 04:23 | |
| *** jcooley_ has joined #openstack-infra | 04:26 | |
| *** talluri has quit IRC | 04:27 | |
| *** yamahata has joined #openstack-infra | 04:30 | |
| *** michchap has quit IRC | 04:31 | |
| *** jcooley_ has quit IRC | 04:32 | |
| *** jcooley_ has joined #openstack-infra | 04:33 | |
| *** michchap has joined #openstack-infra | 04:37 | |
| *** pcrews has joined #openstack-infra | 04:38 | |
| openstackgerrit | A change was merged to openstack-dev/hacking: Remove vim modeline sample https://review.openstack.org/55890 | 04:39 |
| *** xchu has quit IRC | 04:41 | |
| *** fungi has quit IRC | 04:53 | |
| *** wenlock has joined #openstack-infra | 04:56 | |
| *** wenlock has quit IRC | 04:58 | |
| openstackgerrit | A change was merged to openstack/requirements: Updated from global requirements https://review.openstack.org/55861 | 05:17 |
| *** senk has quit IRC | 05:20 | |
| *** mkoderer has joined #openstack-infra | 05:27 | |
| *** senk has joined #openstack-infra | 05:33 | |
| *** talluri_ has quit IRC | 05:37 | |
| *** sarob has joined #openstack-infra | 05:43 | |
| *** senk has quit IRC | 05:44 | |
| *** pcrews has quit IRC | 05:50 | |
| *** talluri has joined #openstack-infra | 05:52 | |
| *** nicedice_ has quit IRC | 05:54 | |
| *** ljjjusti1 has quit IRC | 05:58 | |
| *** ljjjustin has joined #openstack-infra | 05:59 | |
| *** markwash has joined #openstack-infra | 06:03 | |
| *** nati_ueno has joined #openstack-infra | 06:07 | |
| *** michchap has quit IRC | 06:10 | |
| *** michchap has joined #openstack-infra | 06:11 | |
| *** EmilienM has quit IRC | 06:13 | |
| *** sdague has quit IRC | 06:13 | |
| *** Guest53429 has quit IRC | 06:15 | |
| *** hughsaunders has quit IRC | 06:15 | |
| *** Adri2000 has quit IRC | 06:16 | |
| *** Adri2000 has joined #openstack-infra | 06:17 | |
| *** wenlock has joined #openstack-infra | 06:17 | |
| *** sdague has joined #openstack-infra | 06:18 | |
| *** EmilienM has joined #openstack-infra | 06:18 | |
| *** DennyZhang has joined #openstack-infra | 06:25 | |
| *** nati_uen_ has joined #openstack-infra | 06:28 | |
| *** hughsaunders has joined #openstack-infra | 06:28 | |
| *** nati_ueno has quit IRC | 06:31 | |
| *** moted has quit IRC | 06:32 | |
| *** ljjjustin has quit IRC | 06:32 | |
| *** hughsaunders has quit IRC | 06:33 | |
| *** sarob has quit IRC | 06:35 | |
| *** sarob has joined #openstack-infra | 06:35 | |
| *** fifieldt has joined #openstack-infra | 06:38 | |
| *** sarob has quit IRC | 06:40 | |
| *** pblaho has joined #openstack-infra | 06:43 | |
| *** senk has joined #openstack-infra | 06:45 | |
| *** hughsaunders has joined #openstack-infra | 06:46 | |
| *** nati_ueno has joined #openstack-infra | 06:48 | |
| *** nati_uen_ has quit IRC | 06:49 | |
| *** hughsaunders has quit IRC | 06:50 | |
| *** senk has quit IRC | 06:52 | |
| *** fungi has joined #openstack-infra | 06:55 | |
| *** nati_ueno has quit IRC | 06:57 | |
| *** nati_ueno has joined #openstack-infra | 06:58 | |
| *** talluri_ has joined #openstack-infra | 06:59 | |
| *** talluri has quit IRC | 06:59 | |
| *** DennyZhang has quit IRC | 07:01 | |
| *** nati_ueno has quit IRC | 07:03 | |
| *** nati_ueno has joined #openstack-infra | 07:04 | |
| *** nati_ueno has quit IRC | 07:09 | |
| *** nati_ueno has joined #openstack-infra | 07:10 | |
| *** hughsaunders has joined #openstack-infra | 07:27 | |
| *** wenlock has quit IRC | 07:28 | |
| *** hughsaunders has quit IRC | 07:33 | |
| *** pblaho has quit IRC | 07:34 | |
| *** sarob has joined #openstack-infra | 07:36 | |
| *** matsuhashi has quit IRC | 07:40 | |
| *** sarob has quit IRC | 07:40 | |
| *** matsuhashi has joined #openstack-infra | 07:41 | |
| *** yolanda has joined #openstack-infra | 07:44 | |
| *** jhesketh_ has quit IRC | 07:45 | |
| *** matsuhashi has quit IRC | 07:45 | |
| *** matsuhashi has joined #openstack-infra | 07:46 | |
| *** senk has joined #openstack-infra | 07:48 | |
| *** marun has joined #openstack-infra | 07:53 | |
| *** senk has quit IRC | 07:54 | |
| *** nati_uen_ has joined #openstack-infra | 07:58 | |
| *** nati_uen_ has quit IRC | 07:58 | |
| *** nati_uen_ has joined #openstack-infra | 07:59 | |
| *** nati_ueno has quit IRC | 08:01 | |
| *** lillie has joined #openstack-infra | 08:03 | |
| *** hughsaunders has joined #openstack-infra | 08:03 | |
| *** lillie is now known as Guest27104 | 08:03 | |
| *** senk has joined #openstack-infra | 08:10 | |
| openstackgerrit | Masayuki Igawa proposed a change to openstack-infra/elastic-recheck: Add query for bug 1249535 https://review.openstack.org/55982 | 08:11 |
| *** jcooley_ has quit IRC | 08:14 | |
| *** senk has quit IRC | 08:15 | |
| *** osanchez has joined #openstack-infra | 08:17 | |
| *** yamahata has quit IRC | 08:25 | |
| *** davidhadas_ has joined #openstack-infra | 08:30 | |
| *** davidhadas has quit IRC | 08:31 | |
| *** nati_uen_ has quit IRC | 08:36 | |
| *** sarob has joined #openstack-infra | 08:36 | |
| *** nati_uen_ has joined #openstack-infra | 08:37 | |
| *** nati_uen_ has quit IRC | 08:38 | |
| *** nati_uen_ has joined #openstack-infra | 08:38 | |
| *** shardy_afk is now known as shardy | 08:40 | |
| *** bingbu has quit IRC | 08:41 | |
| *** sarob has quit IRC | 08:41 | |
| *** jpich has joined #openstack-infra | 08:41 | |
| *** nati_uen_ has quit IRC | 08:43 | |
| *** davidhadas_ has quit IRC | 08:45 | |
| *** yassine has joined #openstack-infra | 08:54 | |
| *** briancline has quit IRC | 08:56 | |
| *** EmilienM has quit IRC | 08:56 | |
| *** briancline has joined #openstack-infra | 08:56 | |
| *** EmilienM has joined #openstack-infra | 08:56 | |
| *** nosnos_ has joined #openstack-infra | 08:57 | |
| *** fifieldt has quit IRC | 08:58 | |
| *** hashar has joined #openstack-infra | 08:59 | |
| *** nosnos has quit IRC | 09:01 | |
| *** yolanda has quit IRC | 09:07 | |
| *** yolanda has joined #openstack-infra | 09:07 | |
| *** hdd has joined #openstack-infra | 09:09 | |
| *** senk has joined #openstack-infra | 09:11 | |
| *** derekh has joined #openstack-infra | 09:12 | |
| *** nati_ueno has joined #openstack-infra | 09:14 | |
| *** senk has quit IRC | 09:15 | |
| *** pblaho has joined #openstack-infra | 09:18 | |
| *** ben_duyujie has joined #openstack-infra | 09:18 | |
| *** johnthetubaguy has joined #openstack-infra | 09:33 | |
| *** plomakin has quit IRC | 09:36 | |
| *** nati_ueno has quit IRC | 09:36 | |
| *** plomakin has joined #openstack-infra | 09:37 | |
| *** sarob has joined #openstack-infra | 09:37 | |
| *** nati_ueno has joined #openstack-infra | 09:37 | |
| *** johnthetubaguy1 has joined #openstack-infra | 09:38 | |
| *** johnthetubaguy has quit IRC | 09:39 | |
| *** marun has quit IRC | 09:41 | |
| *** sarob has quit IRC | 09:41 | |
| *** matsuhashi has quit IRC | 09:41 | |
| *** nati_ueno has quit IRC | 09:42 | |
| *** matsuhashi has joined #openstack-infra | 09:42 | |
| *** matsuhas_ has joined #openstack-infra | 09:44 | |
| *** matsuhashi has quit IRC | 09:45 | |
| *** pleia2 has quit IRC | 10:02 | |
| *** matsuhas_ has quit IRC | 10:02 | |
| *** pleia2 has joined #openstack-infra | 10:03 | |
| *** matsuhashi has joined #openstack-infra | 10:03 | |
| openstackgerrit | Russell Bryant proposed a change to openstack-infra/reviewstats: Add some core team review stats https://review.openstack.org/55995 | 10:06 |
| openstackgerrit | Russell Bryant proposed a change to openstack-infra/reviewstats: Show how many patch sets were created https://review.openstack.org/55996 | 10:06 |
| *** matsuhashi has quit IRC | 10:07 | |
| russellb | lifeless: want to join stats-core so you can help review reviewstats changes? | 10:09 |
| lifeless | russellb: if you would like me to do such reviews, I'd be happy to do so. | 10:09 |
| lifeless | russellb: [yes] | 10:09 |
| russellb | yeah, you've shown interest and i trust you :-) ... i should probably stop self-approving everything | 10:10 |
| lifeless | Though you won't get reviews tonight, its past 11 and I'm wrapping up my day :) | 10:10 |
| *** yamahata has joined #openstack-infra | 10:10 | |
| russellb | no worries, just asking while i thought of it | 10:10 |
| russellb | (done) | 10:10 |
| lifeless | cool | 10:10 |
| *** senk has joined #openstack-infra | 10:13 | |
| *** senk has quit IRC | 10:17 | |
| *** markmc has joined #openstack-infra | 10:18 | |
| *** guohliu has quit IRC | 10:19 | |
| *** ekarlso has quit IRC | 10:20 | |
| *** ekarlso has joined #openstack-infra | 10:21 | |
| *** matsuhashi has joined #openstack-infra | 10:28 | |
| openstackgerrit | Russell Bryant proposed a change to openstack-infra/reviewstats: Add submitted patches to review stats https://review.openstack.org/55999 | 10:32 |
| *** sarob has joined #openstack-infra | 10:37 | |
| *** adalbas has joined #openstack-infra | 10:41 | |
| *** yamahata has quit IRC | 10:42 | |
| *** sarob has quit IRC | 10:42 | |
| *** yaguang has quit IRC | 10:45 | |
| *** nati_ueno has joined #openstack-infra | 10:47 | |
| *** nati_ueno has quit IRC | 10:52 | |
| *** rfolco has joined #openstack-infra | 11:03 | |
| *** lcestari has joined #openstack-infra | 11:03 | |
| *** nosnos_ has quit IRC | 11:08 | |
| *** nosnos has joined #openstack-infra | 11:09 | |
| *** senk has joined #openstack-infra | 11:14 | |
| *** hashar has quit IRC | 11:17 | |
| *** hashar has joined #openstack-infra | 11:17 | |
| *** matsuhashi has quit IRC | 11:18 | |
| *** senk has quit IRC | 11:19 | |
| *** Reapster has quit IRC | 11:19 | |
| *** matsuhashi has joined #openstack-infra | 11:30 | |
| *** hashar has quit IRC | 11:37 | |
| *** sarob has joined #openstack-infra | 11:38 | |
| *** yamahata has joined #openstack-infra | 11:41 | |
| *** yamahata has quit IRC | 11:42 | |
| *** sarob has quit IRC | 11:42 | |
| *** yamahata has joined #openstack-infra | 11:43 | |
| *** michchap has quit IRC | 11:43 | |
| *** pcm_ has joined #openstack-infra | 11:45 | |
| *** yfried has joined #openstack-infra | 11:46 | |
| *** michchap has joined #openstack-infra | 11:46 | |
| yfried | anyone knows if during Jenkins' review the VMs should have external access or not? ie - be able to ping 8.8.8.8 or www.google.com? | 11:47 |
| *** pcm_ has quit IRC | 11:48 | |
| *** pcm_ has joined #openstack-infra | 11:49 | |
| *** ArxCruz has joined #openstack-infra | 11:50 | |
| *** CaptTofu has joined #openstack-infra | 11:57 | |
| *** hdd has quit IRC | 12:00 | |
| *** alcabrera has joined #openstack-infra | 12:02 | |
| *** matsuhashi has quit IRC | 12:05 | |
| *** matsuhashi has joined #openstack-infra | 12:06 | |
| *** nosnos_ has joined #openstack-infra | 12:07 | |
| *** nosnos has quit IRC | 12:10 | |
| *** matsuhashi has quit IRC | 12:10 | |
| *** nosnos_ has quit IRC | 12:11 | |
| *** yfried has left #openstack-infra | 12:13 | |
| *** michchap has quit IRC | 12:13 | |
| *** michchap has joined #openstack-infra | 12:13 | |
| *** yaguang has joined #openstack-infra | 12:15 | |
| *** senk has joined #openstack-infra | 12:15 | |
| *** senk has quit IRC | 12:20 | |
| anteaya | hi yfried, no vms do not have external access - they are disposable - brought up for the tests and then after the tests are complete and logs are written to logs.openstack.org, they are brought down | 12:26 |
| *** marun has joined #openstack-infra | 12:28 | |
| *** CaptTofu has quit IRC | 12:28 | |
| *** CaptTofu has joined #openstack-infra | 12:29 | |
| *** beagles has joined #openstack-infra | 12:31 | |
| *** weshay has joined #openstack-infra | 12:33 | |
| *** CaptTofu has quit IRC | 12:33 | |
| *** dcramer_ has quit IRC | 12:35 | |
| *** che-arne has joined #openstack-infra | 12:35 | |
| *** sarob has joined #openstack-infra | 12:38 | |
| *** sarob has quit IRC | 12:43 | |
| *** afazekas has joined #openstack-infra | 12:46 | |
| *** dcramer_ has joined #openstack-infra | 12:50 | |
| *** yassine has quit IRC | 12:54 | |
| *** yassine has joined #openstack-infra | 12:54 | |
| *** bingbu has joined #openstack-infra | 12:56 | |
| mordred | anteaya: not quite. they _do_ have external access - but they are disposable | 12:58 |
| openstackgerrit | Cyril Roelandt proposed a change to openstack-infra/config: Add sphinxcontrib-docbookrestapi https://review.openstack.org/53306 | 13:01 |
| anteaya | ah okay, glad to be wrong on that one | 13:04 |
| anteaya | how does a developer access a vm running a test, mordred | 13:04 |
| anteaya | I thought that functionality was refused from the cloud providers we are using | 13:05 |
| *** afazekas has quit IRC | 13:07 | |
| *** oubiwann has joined #openstack-infra | 13:11 | |
| mordred | anteaya: that they cannot do | 13:11 |
| mordred | but the vm can make _outbound_ connections | 13:11 |
| *** mriedem has joined #openstack-infra | 13:12 | |
| anteaya | ah okay, so I took one definition of external access and did not consider all the possible meanings | 13:13 |
| anteaya | yes, they would need outbound connections to download the repos to stand up the devstack to run the tests | 13:14 |
| anteaya | thanks mordred | 13:14 |
| anteaya | and good morning | 13:14 |
| *** jorisroovers has joined #openstack-infra | 13:16 | |
| *** thomasem has joined #openstack-infra | 13:17 | |
| mordred | good morning anteaya | 13:18 |
| *** dims has joined #openstack-infra | 13:19 | |
| *** hashar has joined #openstack-infra | 13:20 | |
| pleia2 | good evening! | 13:21 |
| *** flaper87|afk is now known as flaper87 | 13:23 | |
| *** weshay has quit IRC | 13:23 | |
| anteaya | morning pleia2 | 13:25 |
| anteaya | goodness are you up early | 13:25 |
| anteaya | ah you said good evening | 13:25 |
| anteaya | in which case are you up late | 13:25 |
| pleia2 | coming home is rough times :) | 13:25 |
| anteaya | no kidding | 13:28 |
| anteaya | I was up for most of the night, night before last | 13:29 |
| anteaya | up at 6am this morning | 13:29 |
| anteaya | did some laundry | 13:29 |
| anteaya | so that was good | 13:29 |
| *** Adri2000 has quit IRC | 13:30 | |
| *** Adri2000 has joined #openstack-infra | 13:31 | |
| *** changbl has quit IRC | 13:36 | |
| openstackgerrit | Nikita Konovalov proposed a change to openstack-infra/storyboard: Added task ordering https://review.openstack.org/56026 | 13:38 |
| openstackgerrit | Jeffrey Zhang proposed a change to openstack-dev/hacking: Fix the escape character https://review.openstack.org/56028 | 13:42 |
| *** che-arne has quit IRC | 13:44 | |
| *** changbl has joined #openstack-infra | 13:47 | |
| *** pblaho has quit IRC | 13:49 | |
| sdague | morning folks | 13:56 |
| sdague | man, actually got 8 hrs sleep last night for the first time since before HK, though in 2 4hr chunks. Maybe I'll sleep through the night tonight :) | 13:56 |
| dims | sdague, nice. btw we are getting a dusting of snow today here | 13:58 |
| sdague | yeh, here too, I twittered the view this morning :) | 13:59 |
| sdague | pleia2, were you also in the trove integration test session that jeblair called me into half way through? | 14:00 |
| pleia2 | sdague: yeah | 14:00 |
| *** dprince has joined #openstack-infra | 14:01 | |
| pleia2 | my blog tells me that much time was spent talking about image handling (or at least that's all I paid attention to) | 14:01 |
| *** freyes has joined #openstack-infra | 14:01 | |
| sdague | so I had an interesting call with some internal folks yesterday that have previously built simulators for cloud like environments | 14:01 |
| pleia2 | cool | 14:01 |
| sdague | they basically used honeyd to simulate real guests | 14:02 |
| pleia2 | I've never even heard of honeyd | 14:02 |
| sdague | so you could have ip services without real guests | 14:02 |
| sdague | it's a honey pot system | 14:02 |
| *** Adri2000 has quit IRC | 14:02 | |
| pleia2 | huh, interesting | 14:02 |
| sdague | so I'm hoping that they engage the community directly come January, that looks like their plan, but it's really a pretty cool idea | 14:03 |
| *** talluri_ has quit IRC | 14:03 | |
| sdague | because it means we could have fake virt driver that actually built fake guests you could ping and ssh to | 14:03 |
| *** dkranz has quit IRC | 14:03 | |
| mordred | neat | 14:04 |
| pleia2 | yeah, that would be great | 14:04 |
| *** talluri has joined #openstack-infra | 14:04 | |
| mordred | jog0: ^^ | 14:04 |
| sdague | I thought it was a pretty clever approach | 14:04 |
| *** wenlock has joined #openstack-infra | 14:04 | |
| *** davidhadas has joined #openstack-infra | 14:04 | |
| sdague | they also want to do fault injection and delay fuzzing in the fake driver, which would also be really good. | 14:04 |
| *** dcramer_ has quit IRC | 14:06 | |
| *** che-arne has joined #openstack-infra | 14:06 | |
| *** marun has quit IRC | 14:07 | |
| *** talluri has quit IRC | 14:08 | |
| *** sarob has joined #openstack-infra | 14:09 | |
| *** dstanek has joined #openstack-infra | 14:09 | |
| *** oubiwann has quit IRC | 14:12 | |
| *** julim has joined #openstack-infra | 14:13 | |
| mordred | that's truly fantastic | 14:13 |
| *** flaper87 is now known as flaper87|afk | 14:13 | |
| *** sarob has quit IRC | 14:13 | |
| *** guohliu has joined #openstack-infra | 14:14 | |
| *** oubiwann has joined #openstack-infra | 14:16 | |
| *** sandywalsh has joined #openstack-infra | 14:17 | |
| *** changbl has quit IRC | 14:17 | |
| sdague | yep, agreed. So anyway, I'm hopeful they'll fully engage. | 14:18 |
| *** Adri2000 has joined #openstack-infra | 14:20 | |
| *** Adri2000 has joined #openstack-infra | 14:20 | |
| *** dkliban has joined #openstack-infra | 14:23 | |
| *** weshay has joined #openstack-infra | 14:24 | |
| *** weshay has quit IRC | 14:24 | |
| mordred | ttx: we have folks starting to actively hack on storyboard, and I think I may be the only person really reviewing - any thoughts on what to do about making/growing the initial core team? | 14:26 |
| *** weshay has joined #openstack-infra | 14:26 | |
| mordred | ttx: other early projects seem to just add anyone who is initially interested until there is actually a team that's working | 14:27 |
| *** yassine has quit IRC | 14:27 | |
| *** oubiwann has quit IRC | 14:27 | |
| ttx | mordred: I'll be reviewing too... but agree that 3+ would be good | 14:28 |
| ttx | kinda wanted people to at least author one successful patch before getting +2 | 14:28 |
| mordred | ttx: k. I'm going to say that it already looks like nikita might be a good candidate :) | 14:28 |
| mordred | ttx: and yeah | 14:28 |
| *** alexpilotti has joined #openstack-infra | 14:28 | |
| ttx | mordred: will review his patch and fasttrack him in if that makes any sense | 14:29 |
| mordred | ttx: I believe he seems to know more django than you or I at least :) | 14:29 |
| ttx | just want to make sure the person grokked the intent and the code structure. Sure looks like nikita does | 14:29 |
| *** ben_duyujie has quit IRC | 14:29 | |
| ttx | reviewing his patch are next on my list | 14:30 |
| ttx | currently making big lists of all the stuff I signed up for last week | 14:30 |
| ttx | what was I exactly thinking, I don't know | 14:30 |
| mordred | ttx: tell me about it | 14:30 |
| *** alexpilotti has quit IRC | 14:32 | |
| *** bingbu has quit IRC | 14:34 | |
| mordred | lifeless: I've made you core on pbr. go nuts | 14:34 |
| *** jroovers has joined #openstack-infra | 14:38 | |
| *** jorisroovers has quit IRC | 14:41 | |
| *** yassine has joined #openstack-infra | 14:43 | |
| *** dstanek has quit IRC | 14:49 | |
| mordred | clarkb, fungi: when you are up and human, https://review.openstack.org/#/c/54791/ could really use some love | 14:49 |
| *** dkranz has joined #openstack-infra | 14:50 | |
| mordred | as could https://review.openstack.org/#/c/54818/ | 14:50 |
| mordred | the second one is actually more important, but bother would be excellent steps forwrad | 14:50 |
| *** CaptTofu has joined #openstack-infra | 14:51 | |
| *** dstanek has joined #openstack-infra | 14:52 | |
| *** oubiwann has joined #openstack-infra | 14:53 | |
| openstackgerrit | A change was merged to openstack-dev/hacking: Fix the escape character https://review.openstack.org/56028 | 14:55 |
| dprince | mordred: can you add me and markmc as members of the TripleO org on Trello? | 14:58 |
| dprince | mordred: https://trello.com/tripleo | 14:58 |
| *** jpeeler has quit IRC | 14:58 | |
| dprince | mordred: or perhaps just make derekh an admin there? (he is core) | 14:58 |
| *** jpeeler has joined #openstack-infra | 14:59 | |
| *** dcramer_ has joined #openstack-infra | 15:00 | |
| *** sarob has joined #openstack-infra | 15:00 | |
| *** guohliu has quit IRC | 15:04 | |
| *** rwsu-away is now known as rwsu | 15:04 | |
| mordred | (moving above to #tripleo) | 15:05 |
| *** oubiwann has quit IRC | 15:09 | |
| *** boris-42 has joined #openstack-infra | 15:11 | |
| *** oubiwann has joined #openstack-infra | 15:13 | |
| *** openstackstatus has quit IRC | 15:13 | |
| *** rnirmal has joined #openstack-infra | 15:14 | |
| anteaya | reed: when you are around my foundation profile says I have been affiliated with 175game: http://www.openstack.org/community/members/profile/7571 | 15:14 |
| anteaya | I have no idea who/what 175game is/are | 15:14 |
| anteaya | but now I am curious | 15:14 |
| fungi | anteaya: reed seems to be not around, but i'll prod toddmorey via e-mail | 15:17 |
| fungi | i suspect they're all screwy. mine says "ludio333 From To 2013-03-21 ... IB3 Televisió de les Illes Balears From To 2013-03-21 ... ludio333 From To 2013-03-21" | 15:18 |
| fungi | probably someone fat-fingered a db query | 15:18 |
| anteaya | ha ha ha | 15:18 |
| anteaya | did they pay you well is my question | 15:19 |
| mordred | wow. seriously? | 15:19 |
| mordred | someone is running db queries that they can fat finger? | 15:19 |
| anteaya | 175game and ludio333 feel like auto-generated names to me | 15:20 |
| anteaya | perhaps someone was testing something and didn't realize they were on production | 15:20 |
| * mordred sobs | 15:20 | |
| mordred | fungi: could you please communicate to your boss that I'm 3 seconds from absolutely losing my shit on him | 15:20 |
| *** hashar has quit IRC | 15:21 | |
| mordred | fungi: and that, respectfully, if they don't get their shit moved over ASAP, I'm going to do it publically. very. very. publically. | 15:21 |
| fungi | mordred: i do have a call with them all this evening. i'll do my best to convey that | 15:22 |
| fungi | i feel very much the same way | 15:22 |
| ttx | mordred: they had some excuse due to release/summit shit... but now they should definitely commit on a due date for delivering | 15:23 |
| mordred | ttx: all of their excuses are bullshit | 15:23 |
| *** senk1 has joined #openstack-infra | 15:23 | |
| mordred | ttx: we started asking for this a YEAR ago | 15:23 |
| anteaya | I'd log in a fix my foundation profile but I don't have the energy yet to play find the foundation login link and remember (or create a new) password | 15:23 |
| mordred | ttx: it is, and has been, categorically unacceptable that this has gone on for as long as it has | 15:24 |
| mordred | that there was a summit is completely immaterial | 15:24 |
| ttx | mordred: I think, deep in them they don't really want to lose control over this, so they unconsciously do something else rather than that. | 15:24 |
| mordred | in the course of that year, there have been mulitple instances of site being down, dns being broken | 15:24 |
| mordred | and now apparently database corruption | 15:24 |
| mordred | they are not competent to run it | 15:24 |
| mordred | they are repeatedly showing that incompetence | 15:25 |
| mordred | and they are, for reasons surpassing all understanding, balking at particpating in the systems that run the fastest growing open source project of all time, of which they are the foundatoin that supports it | 15:25 |
| mordred | so | 15:25 |
| mordred | seriously | 15:25 |
| mordred | it is assinine and arrogant | 15:25 |
| mordred | and I've had enough | 15:25 |
| ttx | fungi: I may skip the midnight meeting today, so I'll let you convey that things are about to get bad and they better deliver | 15:26 |
| * ttx has a backlog of stuff, like a proper security page, to push to o.o | 15:27 | |
| ttx | mordred: about those storyboard reviews, I feel a bit bad for Nikita, as the UI stuff heavily depends on the tech we'll end up using, and we are about to seriously question the one used in the POC :) | 15:28 |
| fungi | yeah, there's so much that we've said "let's wait until we have access to the www.o.o site source so we can just submit patches for this or that" | 15:29 |
| mordred | ttx: I think we do that in the context of the project that exists | 15:29 |
| anteaya | ttx where are the storyboard conversations happening? | 15:29 |
| ttx | anteaya: here, so far | 15:29 |
| anteaya | okay | 15:29 |
| anteaya | I have some input on the tech when the conversation happens | 15:29 |
| *** david-lyle has joined #openstack-infra | 15:30 | |
| anteaya | an extention or clarification of what I shared at the summit | 15:30 |
| ttx | mordred: agreed, I just fear that some of that jus(submitted code might be scrapped soon | 15:30 |
| ttx | mordred: but meh | 15:30 |
| mordred | if we do crazy/radical things, I think they will be follow-on patches to that repo, no? | 15:31 |
| ttx | mordred: oh, sure | 15:33 |
| ttx | it's just that pagination/sorting stuff is typically the sort of work that depend a lot on the UI/Model/tech used | 15:34 |
| ttx | some framework give it for free, some others not etc. | 15:35 |
| ttx | can be done client-side or server-side etc | 15:35 |
| mordred | are we considering wholesale replacement of hte UI framework now? | 15:35 |
| ttx | (precise reason why i didn't look into it just yet :) | 15:35 |
| ttx | mordred: cody suggested we switch to semantic ui. Also a REST + horizon layer switch could affect sorting/paging stuff | 15:37 |
| ttx | anyway, it's still interesting to have those landed in the current UI | 15:37 |
| *** markmcclain has joined #openstack-infra | 15:37 | |
| *** datsun180b has joined #openstack-infra | 15:37 | |
| *** senk1 has quit IRC | 15:38 | |
| cody-somerville | anteaya: What's your input? | 15:38 |
| *** lcestari has quit IRC | 15:39 | |
| *** lcestari has joined #openstack-infra | 15:43 | |
| *** lcestari has quit IRC | 15:43 | |
| *** senk has joined #openstack-infra | 15:45 | |
| *** alcabrera has quit IRC | 15:48 | |
| openstackgerrit | A change was merged to openstack-infra/config: Add sphinxcontrib-docbookrestapi https://review.openstack.org/53306 | 15:49 |
| mordred | BOOM | 15:49 |
| mordred | fungi, clarkb: watch out - I'm second on the review list AND my approval percent is down to 80 | 15:50 |
| fungi | mordred: i haven't looked but can pretty well guess i've fallen waaay down the list in the past couple weeks anyway | 15:51 |
| mordred | fungi: well, we've ALL been busy | 15:51 |
| mordred | you're #3 now | 15:51 |
| fungi | oh, uh | 15:51 |
| fungi | okay then | 15:51 |
| mordred | :) | 15:51 |
| mordred | but- lifeless is beating jeblair | 15:51 |
| * fungi feels perpetually as if he doesn't get enough time to review anything at all | 15:51 | |
| mordred | I think that _might_ have something to do with jeblair being on vacation though | 15:52 |
| openstackgerrit | A change was merged to openstack-infra/gitdm: Fix broken email mapping and add missing domains https://review.openstack.org/53419 | 15:52 |
| mordred | fungi: I've started doing them first thing when I wake | 15:52 |
| fungi | mordred: no excuse. just think of all the reviewing he could get done with multiple weeks of vacation | 15:52 |
| mordred | and my rule is that I am not allowed to write any new patches until I've reviewed everything in my queue | 15:52 |
| fungi | that's actually a great policy | 15:52 |
| fungi | i think i'll adopt it... right after this next patch ;) | 15:53 |
| mordred | then, I try to review things as I see them roll past in IRC during hte day | 15:53 |
| openstackgerrit | DennyZhang proposed a change to openstack-infra/config: remove code duplication in run-*.sh https://review.openstack.org/54646 | 15:53 |
| openstackgerrit | Kevin proposed a change to openstack-infra/jenkins-job-builder: Added template-test command & run-after-finalized https://review.openstack.org/54993 | 15:54 |
| mordred | fungi: getting the queue down post summit took me about 4 hours | 15:54 |
| mordred | BUT - I +2/+A'd a bunch of trivial stuff, so your queue may not be as terrible | 15:54 |
| *** senk has quit IRC | 15:55 | |
| *** zehicle_at_dell has joined #openstack-infra | 15:55 | |
| *** jamesmcarthur has joined #openstack-infra | 15:55 | |
| *** smarcet has joined #openstack-infra | 15:55 | |
| *** jroovers has quit IRC | 15:55 | |
| *** senk has joined #openstack-infra | 15:56 | |
| *** sarob has quit IRC | 15:57 | |
| *** sparkycollier has joined #openstack-infra | 15:57 | |
| *** sarob has joined #openstack-infra | 15:57 | |
| *** rcleere has joined #openstack-infra | 15:58 | |
| anteaya | cody-somerville: that django is a keeper for obvious reasons and that a framework for the frontend locks us into its feature set unless it comes with vendor passthrough, which I didn't see in the current tool being used | 16:00 |
| *** senk has quit IRC | 16:01 | |
| anteaya | if it is there, I couldn't find it | 16:01 |
| *** mfer has joined #openstack-infra | 16:01 | |
| *** sarob has quit IRC | 16:02 | |
| *** briancline has quit IRC | 16:02 | |
| mordred | anteaya: can you explain "vendor passthrough" ? | 16:03 |
| *** toddmorey has joined #openstack-infra | 16:03 | |
| *** ^d has joined #openstack-infra | 16:03 | |
| *** pcrews has joined #openstack-infra | 16:03 | |
| *** adalbas has quit IRC | 16:08 | |
| *** UtahDave has joined #openstack-infra | 16:08 | |
| olaph | sounds like what you get from eating at the state fair | 16:09 |
| openstackgerrit | Monty Taylor proposed a change to openstack-dev/pbr: Base python 2.7 skip list on parsed names https://review.openstack.org/54152 | 16:10 |
| *** lcestari has joined #openstack-infra | 16:12 | |
| *** briancline has joined #openstack-infra | 16:13 | |
| anteaya | mordred: sorry, I was thinking in ironic nomenclature - I mean that the framework has a way of colouring different text for example, but if I want to select the text in a different way, I don't have the option of just writing my own css and sticking it in a file | 16:16 |
| anteaya | at least not that I saw | 16:16 |
| anteaya | I am prepared to be wrong | 16:17 |
| openstackgerrit | Monty Taylor proposed a change to openstack-dev/pbr: Make ChangeLog more condensed https://review.openstack.org/52367 | 16:19 |
| *** adalbas has joined #openstack-infra | 16:21 | |
| *** yaguang has quit IRC | 16:23 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/nodepool: Add ability to filter on flavor name https://review.openstack.org/55844 | 16:25 |
| *** sarob has joined #openstack-infra | 16:28 | |
| *** nicedice_ has joined #openstack-infra | 16:29 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Fix pep8 errors in modules dir https://review.openstack.org/47644 | 16:30 |
| *** CaptTofu has quit IRC | 16:31 | |
| *** yamahata has quit IRC | 16:31 | |
| *** CaptTofu has joined #openstack-infra | 16:31 | |
| *** boris-42 has quit IRC | 16:32 | |
| *** sarob has quit IRC | 16:34 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Switch from jeepyb to pypi-mirror https://review.openstack.org/54818 | 16:34 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Remove old pypi host cruft https://review.openstack.org/54817 | 16:34 |
| *** sarob has joined #openstack-infra | 16:34 | |
| *** jergerber has joined #openstack-infra | 16:37 | |
| *** sarob_ has joined #openstack-infra | 16:37 | |
| *** sarob has quit IRC | 16:38 | |
| *** sarob_ has quit IRC | 16:38 | |
| *** ruhe has joined #openstack-infra | 16:38 | |
| *** sarob has joined #openstack-infra | 16:38 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-dev/pbr: Ignore boring commit messages https://review.openstack.org/54048 | 16:39 |
| openstackgerrit | Monty Taylor proposed a change to openstack-dev/pbr: Let git do mailmap mapping for us https://review.openstack.org/54050 | 16:39 |
| *** sarob_ has joined #openstack-infra | 16:40 | |
| *** changbl has joined #openstack-infra | 16:40 | |
| *** markmcclain has quit IRC | 16:41 | |
| *** sarob_ has quit IRC | 16:42 | |
| *** sarob_ has joined #openstack-infra | 16:42 | |
| *** sandywalsh has quit IRC | 16:42 | |
| *** DennyZhang has joined #openstack-infra | 16:43 | |
| *** sarob has quit IRC | 16:43 | |
| *** mrodden1 has quit IRC | 16:43 | |
| *** MarkAtwood has quit IRC | 16:43 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-dev/pbr: Skip the install of pip https://review.openstack.org/54061 | 16:44 |
| toddmorey | Hey guys: I wanted to give a quick update here about Member data. TLDR: there was a bug in a migration script we ran and we are working to correct the presentation of OpenStack Foundation member affiliations. The data's intact, but unfortunately currently displaying incorrectly. | 16:44 |
| *** adalbas has quit IRC | 16:47 | |
| toddmorey | We recently rolled out an update that greatly improves the way affiliations are managed and stored. It was badly needed, but since the schema changed a bit, we needed to migrate data over to the new schema. Seems like a bug in that migration caused some of the foundation members to display the wrong affiliation. The good news is that the data looks complete and uncorrupted. We're working to fix the bug in the script, test, test, test, and rerun. I | 16:47 |
| *** sarob_ has quit IRC | 16:47 | |
| *** CaptTofu has quit IRC | 16:48 | |
| *** CaptTofu has joined #openstack-infra | 16:48 | |
| toddmorey | So I'm pretty bummed that we let a bug get out to production, but I'm happy that it seems we should be able to fix things up quickly. But I apologize for all the confusion it caused. | 16:49 |
| toddmorey | I need to change locations, but I'll be back in just a bit and hang out to answer any questions, etc. | 16:50 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/devstack-gate: Use select-mirror in devstack-gate https://review.openstack.org/48064 | 16:50 |
| jamesmcarthur | I'm also available to answer questions re: the migration | 16:51 |
| *** MarkAtwood has joined #openstack-infra | 16:51 | |
| jamesmcarthur | And additionally bummed about this getting out to production. We're looking to patch and push a fix ASAP | 16:51 |
| mordred | jamesmcarthur, toddmorey: awesome. thanks for the update! | 16:52 |
| *** ryanpetrello has joined #openstack-infra | 16:53 | |
| mordred | jamesmcarthur: you're working with sebastian, yeah? | 16:53 |
| jamesmcarthur | mordred: that's correct | 16:54 |
| mordred | cool, well, I'd like to get his change: https://review.openstack.org/#/c/53644/ landed so we can get you guys a thing there | 16:54 |
| openstackgerrit | Kiall Mac Innes proposed a change to openstack-infra/config: Send designate events for stable/havana to #openstack-dns https://review.openstack.org/56057 | 16:54 |
| mordred | fungi, clarkb ^^ can you guys weigh in on the above? | 16:54 |
| jamesmcarthur | we're anxious to get this out there as well. we've made some great progress on the OpenID piece and look forward to getting it out there for review | 16:55 |
| kiall | clearly mordred is talking about my review, not the one he linked ;) | 16:55 |
| kiall | mordred: good summit? | 16:55 |
| mordred | kiall: yes | 16:55 |
| mordred | kiall: I +A'd your patch, btw | 16:56 |
| mordred | jamesmcarthur: ++ | 16:56 |
| openstackgerrit | A change was merged to openstack-infra/config: Send designate events for stable/havana to #openstack-dns https://review.openstack.org/56057 | 16:57 |
| kiall | Thanks :) | 16:57 |
| *** sandywalsh has joined #openstack-infra | 16:57 | |
| *** adalbas has joined #openstack-infra | 16:58 | |
| fungi | mordred: taking a look. got sidetracked trying to get a displayport hub working under linux/xorg. so bleeding edge i apparently need a transfusion | 16:59 |
| *** changbl has quit IRC | 16:59 | |
| *** mrodden has joined #openstack-infra | 16:59 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Gate current clients on stable branches https://review.openstack.org/41931 | 17:01 |
| jamesmcarthur | mordred: just FYI, we've found the migration bug in the script and are working to correct it | 17:02 |
| *** openstackgerrit has quit IRC | 17:03 | |
| *** yassine has quit IRC | 17:03 | |
| jamesmcarthur | will post back as soon as we've rectified it. again, apologies for the mass confusion on all of this. we've tried to be very careful with this data and our scripts, but this one snuck through the cracks. | 17:03 |
| *** openstackgerrit has joined #openstack-infra | 17:04 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-dev/pbr: Make ChangeLog more condensed https://review.openstack.org/52367 | 17:06 |
| openstackgerrit | Monty Taylor proposed a change to openstack-dev/pbr: Ignore boring commit messages https://review.openstack.org/54048 | 17:06 |
| openstackgerrit | Monty Taylor proposed a change to openstack-dev/pbr: Let git do mailmap mapping for us https://review.openstack.org/54050 | 17:06 |
| mordred | jamesmcarthur: it's all good - it happens to all of us | 17:06 |
| *** toddmorey has quit IRC | 17:07 | |
| mordred | jamesmcarthur: in general, it's one of the reasons we're keen to get you guys integrated, so that we can help with eyeballs and share the blame when we miss it :) | 17:07 |
| jamesmcarthur | absosmurfly | 17:08 |
| jamesmcarthur | more eyeballs is never a bad thing | 17:08 |
| *** gyee has joined #openstack-infra | 17:09 | |
| mordred | they're tasty as well | 17:09 |
| mordred | fungi: can you look at https://review.openstack.org/41931 when you get a chance? I cannot see my error | 17:09 |
| jamesmcarthur | more eyeballs are never a bad thing. terrible grammar. | 17:09 |
| openstackgerrit | Chmouel Boudjnah proposed a change to openstack-infra/config: Add swiftsync. https://review.openstack.org/56061 | 17:11 |
| chmouel | it's the first time i am adding a repo to stackforge is there anything else i need to do to import the github repo ^ ? | 17:12 |
| mordred | chmouel: looking | 17:12 |
| mordred | chmouel: that looks great. it's not going to cut releases to pypi - but you could add that as a follow on | 17:13 |
| chmouel | mordred: cool thanks, yeah we probably want that in the future | 17:13 |
| chmouel | the rtfd ones are for documentation right? | 17:16 |
| *** sandywalsh has quit IRC | 17:21 | |
| *** derekh has quit IRC | 17:25 | |
| fungi | mordred: "Job gate-tempest-devstack-vm-client-stable-grizzly not defined, Job gate-tempest-devstack-vm-client-stable-havana not defined" | 17:26 |
| fungi | mordred: i'll see if i can tell where that's being missed | 17:26 |
| mordred | fungi: I added this: - 'gate-tempest-devstack-vm-client-stable-{release}': | 17:27 |
| fungi | yeah, i'm tracing back through the template instantiation now | 17:27 |
| mordred | chmouel: oh- you're missing an rtfd in projects.yaml | 17:28 |
| mordred | chmouel: look at stackalyitics for an example of using that | 17:28 |
| fungi | mordred: you forgot "pipeline: check" and "pipeline: gate" on the new job entries in projects.yaml | 17:29 |
| fungi | so those aren't making it to the template | 17:29 |
| mordred | fungi: should I change gate-tempest-devstack-vm-client-stable- to {pipeline}-tempest-devstack-vm-client-stable- ? | 17:30 |
| fungi | mordred: yes, that too | 17:31 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Gate current clients on stable branches https://review.openstack.org/41931 | 17:31 |
| mordred | hrm. the others don't do that - lemme try just adding | 17:31 |
| *** ruhe has quit IRC | 17:32 | |
| fungi | the others don't do it because they're only check jobs and so hardcoded in the template | 17:33 |
| fungi | or else they're in the job group so you don't see them enumerated tehre | 17:33 |
| mordred | ah | 17:33 |
| *** miqui has quit IRC | 17:33 | |
| mordred | clarkb: you may want to look at https://review.openstack.org/#/c/54817/ - because it's got a -1 on it so it might not show up inyour todo list | 17:33 |
| dims | mordred, lifeless and i left a couple of comments on the mox/mock question in https://review.openstack.org/#/c/55476/ - i can do either, not a problem | 17:34 |
| fungi | mordred: i reworked those other backwards-compat jobs in there to be check-only since jeblair expressed concerns about breaking our ability to test devstack-gate when the stable branches are perpetually broken | 17:34 |
| mordred | wow. I think I may have taken care of my whole review queue AND outstanding need-work comments | 17:34 |
| *** sandywalsh has joined #openstack-infra | 17:34 | |
| *** pcrews has quit IRC | 17:35 | |
| mordred | dims: oh - I may have been unclear ... | 17:35 |
| mordred | dims: I certainly don't think we should use mox | 17:35 |
| mordred | I was just questioning adding mock for two tests rather than using a feature of the already used fixtures library | 17:36 |
| mordred | dims: but I'm fine with it - I'll go change my vote | 17:36 |
| fungi | mordred: though with all the clients now gating on whether or not stable branches of the integrated servers are working in devstack/tempest, maybe broken stable will get much faster attention anyway | 17:37 |
| dims | mordred, thanks! | 17:38 |
| fungi | either that or we'll start to get a ton of outcry | 17:38 |
| mordred | dims: thank you! | 17:38 |
| *** toddmorey has joined #openstack-infra | 17:39 | |
| mordred | well, a ton of outcry will equal a ton of people to fix the actual problem :) | 17:39 |
| fungi | here's hoping! ;) | 17:39 |
| *** BobBall is now known as BobBallAway | 17:45 | |
| *** dstanek has quit IRC | 17:47 | |
| *** changbl has joined #openstack-infra | 17:55 | |
| clarkb | morning | 17:59 |
| clarkb | mordred: I will take a look | 17:59 |
| *** johnthetubaguy1 has quit IRC | 18:00 | |
| jog0 | pleia2 mordred: neat | 18:02 |
| *** osanchez has quit IRC | 18:03 | |
| *** Ryan_Lane has quit IRC | 18:04 | |
| *** markmcclain has joined #openstack-infra | 18:05 | |
| clarkb | mordred: not sure if I am human yet, but I am taking a look at the reviews you pointed out | 18:05 |
| openstackgerrit | Chmouel Boudjnah proposed a change to openstack-infra/config: Add swiftsync. https://review.openstack.org/56061 | 18:09 |
| clarkb | mordred: fungi https://review.openstack.org/#/c/54818/2 does that change have a sibling that updates the jenkins job configs to use pypi-mirror? | 18:10 |
| clarkb | I just approved the change to move integration.sh to pypi-mirror but the mirror building jobs themselves need to be updated too I think | 18:11 |
| mordred | there is one .. one sec | 18:13 |
| fungi | mordred: clarkb: https://review.openstack.org/54818 | 18:13 |
| clarkb | thanks | 18:14 |
| fungi | though that only installs it on the mirror slaves | 18:15 |
| clarkb | oh that is the change I was looking at | 18:15 |
| clarkb | ya, there should be another change or part of that change that updates JJB configs | 18:15 |
| *** jorisroovers has joined #openstack-infra | 18:15 | |
| mordred | nope. that's all that's needed | 18:16 |
| *** jorisroovers has joined #openstack-infra | 18:16 | |
| mordred | they both install a program called "run-mirror" | 18:16 |
| *** melwitt has joined #openstack-infra | 18:16 | |
| *** dstanek has joined #openstack-infra | 18:16 | |
| fungi | presumably next we need to rip the script out of jeepyb and tag a new release | 18:17 |
| mordred | ++ | 18:17 |
| clarkb | mordred: gotcha | 18:17 |
| fungi | otherwise they're going to fight each other over ownership of the entrypoint | 18:17 |
| fungi | though removing jeepyb from the manifest for those particular servers keeps that from being an issue there i think | 18:18 |
| fungi | but in general, i suspect we'd like to be able to coinstall pypi-mirror and jeepyb on one machine | 18:18 |
| clarkb | fungi: ++ | 18:18 |
| clarkb | I have approved the pypi-mirror stack | 18:20 |
| *** jpich has quit IRC | 18:20 | |
| openstackgerrit | A change was merged to openstack-infra/config: Remove old pypi host cruft https://review.openstack.org/54817 | 18:21 |
| openstackgerrit | Dan Prince proposed a change to openstack-infra/config: Update devstack-cache.py to install RPMs. https://review.openstack.org/54777 | 18:21 |
| openstackgerrit | Dan Prince proposed a change to openstack-infra/config: Add Fedora support to prepare_devstack.sh https://review.openstack.org/54776 | 18:21 |
| openstackgerrit | Dan Prince proposed a change to openstack-infra/config: Add a tempest-devstack-vm-neutron-fedora job. https://review.openstack.org/54779 | 18:21 |
| openstackgerrit | Dan Prince proposed a change to openstack-infra/config: Add Fedora 19 to the nodepool (devstack-f19-check) https://review.openstack.org/54778 | 18:21 |
| openstackgerrit | A change was merged to openstack-infra/config: Switch from jeepyb to pypi-mirror https://review.openstack.org/54818 | 18:21 |
| *** jerryz has joined #openstack-infra | 18:23 | |
| zul | did gerrit go kablooie? | 18:23 |
| clarkb | zul: it is there for me | 18:25 |
| clarkb | zul: what are you seeing? | 18:25 |
| zul | clarkb: my isp just sucks | 18:26 |
| clarkb | (I am hitting it via ipv4 though, perhaps this is an ipv6 problem) | 18:26 |
| clarkb | I am going to double check that puppet runs cleanly on the git.o.o haproxy node and on the jenkins slaves after the postgres and haproxy module updates | 18:27 |
| *** pcrews has joined #openstack-infra | 18:29 | |
| *** rwsu has quit IRC | 18:30 | |
| *** jpich has joined #openstack-infra | 18:34 | |
| *** rwsu has joined #openstack-infra | 18:35 | |
| clarkb | the slaves seem fine with the newer postgres stuff | 18:35 |
| pleia2 | yay | 18:35 |
| clarkb | \o/ that means the work on my end wasn't completely bad :) | 18:35 |
| anteaya | way to go clarkb | 18:36 |
| clarkb | and git.o.o seems fine | 18:36 |
| clarkb | anteaya: this should fix the concat installation issues you had before | 18:36 |
| clarkb | as now everything agrees on a version of concat I think | 18:36 |
| * anteaya thinks back | 18:36 | |
| *** melwitt has quit IRC | 18:36 | |
| *** hogepodge has joined #openstack-infra | 18:37 | |
| anteaya | was this a puppet dashboard issue? | 18:37 |
| *** sarob has joined #openstack-infra | 18:37 | |
| anteaya | the puppet-sql module from way back? | 18:37 |
| clarkb | anteaya: when running install_modules.sh there was a conflict arond the concat module | 18:37 |
| clarkb | because different modules wanted different concat modules. now they all use the same one (or should) | 18:38 |
| *** melwitt has joined #openstack-infra | 18:38 | |
| anteaya | clarkb: ah yes, the module installation | 18:38 |
| anteaya | right | 18:38 |
| anteaya | w00t! | 18:38 |
| anteaya | going to have to test that soon and see | 18:38 |
| clarkb | sdague: fungi: going back to yesterdays swift discussion. I think we should bump the loopback size to 6GB. That will accomodate the default max_file_size in swift | 18:39 |
| clarkb | I am not convinced this will fix our problems but it may | 18:39 |
| *** jpich has quit IRC | 18:40 | |
| clarkb | at the very least it makes the devstack default more sane considering the swift default | 18:40 |
| fungi | agreed, seems worth doing whether or not it actually solves the object creation issue | 18:41 |
| jog0 | silly question: in devstack do we use trunk versions of clients right? | 18:41 |
| *** jpich has joined #openstack-infra | 18:41 | |
| fungi | jog0: yes, and soon also latest releases of them i hope | 18:42 |
| jog0 | fungi: cool, thanks | 18:42 |
| fungi | (this was a takeaway from one of the summit discussions, when it came to light that currently we can actually release a server even though the latest release of the client doesn't work with it, because the master branch since that release may still work) | 18:43 |
| *** krotscheck has joined #openstack-infra | 18:44 | |
| *** markmcclain has quit IRC | 18:44 | |
| *** markmcclain has joined #openstack-infra | 18:46 | |
| ekarlso | is zuul / gerrit / ci system sick atm ? | 18:48 |
| ekarlso | https://review.openstack.org/#/c/55601/ should be +1 and https://review.openstack.org/#/c/55696/ should as well ? | 18:48 |
| clarkb | ekarlso: those appear to be drafts | 18:49 |
| clarkb | or have drafts | 18:49 |
| clarkb | which can't have value'd votes | 18:49 |
| clarkb | I think I contracted that word wrong | 18:49 |
| ekarlso | clarkb: so how to proceed ? | 18:50 |
| clarkb | ekarlso: stop using drafts | 18:51 |
| ekarlso | how to get the change merged though ? | 18:51 |
| clarkb | ekarlso: you need to publish the latest patchset, then you can recheck no bug to have the tests run against that patchset | 18:52 |
| *** ^d has quit IRC | 18:54 | |
| fungi | also, stop using drafts | 18:56 |
| fungi | and, if possible, stop using drafts | 18:56 |
| *** alcabrera has joined #openstack-infra | 18:56 | |
| * fungi really, really hates drafts | 18:56 | |
| clarkb | fungi: after the meeting can you show me what you did to clean up nodepool leaks? I will also take a threaddump then restart nodepool | 18:57 |
| *** markmcclain has quit IRC | 18:57 | |
| clarkb | then we should deal with folsom and havana | 18:57 |
| clarkb | and revoke old keys | 18:57 |
| clarkb | there is so much to do after the summit | 18:57 |
| lifeless | mordred: thanks | 18:57 |
| fungi | clarkb: sure | 18:58 |
| fungi | and yeah, the -folsom/+havana jobs were #2 on my to do list for today, right after submitting my key removal change | 18:58 |
| fungi | so much catch up | 18:58 |
| pleia2 | are we doing a meeting today? | 19:00 |
| clarkb | pleia2: I think we should, we can try to capture summitty stuff after getting through normal agenda | 19:01 |
| pleia2 | ok, meeting is now :) | 19:01 |
| clarkb | yup, silly DST | 19:01 |
| *** jerryz has quit IRC | 19:02 | |
| *** jerryz has joined #openstack-infra | 19:02 | |
| mordred | annegentle: ping | 19:03 |
| annegentle | mordred: ayup | 19:05 |
| annegentle | mordred: I'm up in yer meetin' | 19:05 |
| mordred | annegentle: I'm apparently not paying as much attention to it as I shold be | 19:06 |
| annegentle | mordred: heh | 19:06 |
| *** markwash has quit IRC | 19:07 | |
| *** sarob has quit IRC | 19:08 | |
| *** che-arne has quit IRC | 19:08 | |
| *** sarob has joined #openstack-infra | 19:09 | |
| *** sarob has quit IRC | 19:09 | |
| hogepodge | Last week I branched a stackforge project in anticipation of making releases. I just saw that the branches were merged back into master. | 19:09 |
| hogepodge | Does anyone know why that would happen? | 19:10 |
| *** atiwari has joined #openstack-infra | 19:12 | |
| *** Ryan_Lane has joined #openstack-infra | 19:13 | |
| *** atiwari has quit IRC | 19:14 | |
| *** xeyed4good has joined #openstack-infra | 19:14 | |
| fungi | hogepodge: branches or tags merged back to master? | 19:14 |
| fungi | openstack server projects, which use stable branches, merge tags back to master to keep them linear | 19:15 |
| hogepodge | It says the branches are merged. I haven't tagged the branches yet, and we haven't sent any changes to them. See, for example, https://github.com/stackforge/puppet-openstack/branches?merged=1 | 19:16 |
| clarkb | though that just proposes changes right? | 19:16 |
| hogepodge | or https://github.com/stackforge/puppet-horizon/branches?merged=1 | 19:16 |
| hogepodge | I'm wondering if that will switch back to unmerged when we start pushing changes to the branches? | 19:16 |
| clarkb | hogepodge: if no changes have been made then the branches are identical and github probably sees that as being "merged" | 19:16 |
| hogepodge | Ok, that's what I thought. | 19:17 |
| *** pblaho has joined #openstack-infra | 19:17 | |
| hogepodge | It was a bit surprising at first. "Where did my branches go?" | 19:17 |
| clarkb | hogepodge: basically stop using github :) | 19:17 |
| *** adalbas has quit IRC | 19:18 | |
| *** alcabrera has quit IRC | 19:19 | |
| *** melwitt has quit IRC | 19:21 | |
| *** melwitt1 has joined #openstack-infra | 19:21 | |
| *** ryanpetrello has quit IRC | 19:24 | |
| *** ryanpetrello has joined #openstack-infra | 19:24 | |
| *** aardvark has joined #openstack-infra | 19:25 | |
| *** wusui has quit IRC | 19:28 | |
| *** Protux has joined #openstack-infra | 19:31 | |
| mordred | ++ | 19:40 |
| *** sdake_ is now known as proxy-sdake | 19:45 | |
| *** proxy-sdake is now known as sdake_ | 19:46 | |
| *** markwash has joined #openstack-infra | 19:46 | |
| *** toddmorey1 has joined #openstack-infra | 19:49 | |
| *** sarob has joined #openstack-infra | 19:51 | |
| *** toddmorey has quit IRC | 19:52 | |
| *** ^d has joined #openstack-infra | 19:54 | |
| *** denis_makogon has joined #openstack-infra | 19:56 | |
| *** markmcclain has joined #openstack-infra | 19:56 | |
| *** talluri has joined #openstack-infra | 19:57 | |
| *** talluri has quit IRC | 19:58 | |
| clarkb | I am going to run and grab food quickly. will ping when ready to hit nodepool with a hammer | 20:00 |
| fungi | k | 20:01 |
| *** julim has quit IRC | 20:03 | |
| *** sarob has quit IRC | 20:04 | |
| *** toddmorey1 has quit IRC | 20:05 | |
| *** julim has joined #openstack-infra | 20:06 | |
| *** hashar has joined #openstack-infra | 20:10 | |
| *** lcestari has quit IRC | 20:11 | |
| fungi | also, i have a meeting at 22:00z for an hour, but we should be able to sync up on things prior to that | 20:11 |
| *** jsergent has joined #openstack-infra | 20:11 | |
| anteaya | found out from a kind person helping me correct a flight booking that I need a visa for australia - this was news to me | 20:11 |
| anteaya | apparently folks in the us need one as well | 20:11 |
| anteaya | did others already know this? | 20:11 |
| jsergent | US passport holders need a visa to go to australia? | 20:12 |
| jsergent | you sure about that? | 20:12 |
| jsergent | guess so... | 20:13 |
| jsergent | wow | 20:13 |
| jsergent | http://travel.state.gov/travel/cis_pa_tw/cis/cis_954.html#entry_requirements | 20:13 |
| *** ^d has quit IRC | 20:13 | |
| mordred | anteaya: it's easy | 20:14 |
| mordred | anteaya: you pay like $20 on a website somewhere | 20:14 |
| *** xeyed4good has left #openstack-infra | 20:15 | |
| fungi | and complete a brief quiz about barbecue and beer | 20:15 |
| MarkAtwood | jsergent | 20:15 |
| MarkAtwood | jsergent, yes we do. and the visa expires in just under a year, so i have to keep buying it each LCA | 20:16 |
| MarkAtwood | NZ didnt require the visa, but it does require proof of exit, such as a round trip ticket | 20:16 |
| *** ryanpetrello has quit IRC | 20:17 | |
| anteaya | mordred: was free for me, my agent did it | 20:19 |
| anteaya | just caught be by surprise when she told me it was required | 20:19 |
| jsergent | yeah I heard most airlines do it on behalf of their passengers | 20:19 |
| anteaya | oh and krtaylor it will be great to see ArxCruz at the Atlanta summit | 20:20 |
| ArxCruz | :) | 20:20 |
| anteaya | fungi: so far no questions about barbeque and beer | 20:20 |
| anteaya | ArxCruz: :D | 20:20 |
| anteaya | hope to see you there | 20:21 |
| krtaylor | anteaya, thanks for the reminder! | 20:21 |
| anteaya | krtaylor: :D | 20:22 |
| *** ekarlso has quit IRC | 20:22 | |
| *** ekarlso has joined #openstack-infra | 20:22 | |
| *** yolanda has quit IRC | 20:23 | |
| *** DennyZhang has quit IRC | 20:24 | |
| *** dtroyer has joined #openstack-infra | 20:24 | |
| *** svarnau has joined #openstack-infra | 20:25 | |
| *** dtroyer has quit IRC | 20:26 | |
| *** dtroyer has joined #openstack-infra | 20:27 | |
| *** dtroyer has quit IRC | 20:27 | |
| *** dtroyer has joined #openstack-infra | 20:27 | |
| *** dtroyer has quit IRC | 20:27 | |
| ArxCruz | anteaya: I will do my best to be there :) | 20:32 |
| anteaya | awesome! | 20:33 |
| Mithrandir | oh, good somebody told me about the visa thing, since I didn't need that last time I was at lca | 20:33 |
| Mithrandir | it's free and you can do it online, so I don't really mind, though. | 20:34 |
| *** Hefeweizen has joined #openstack-infra | 20:36 | |
| *** Hefeweizen has quit IRC | 20:36 | |
| *** dtroyer has joined #openstack-infra | 20:37 | |
| *** annegentle has quit IRC | 20:38 | |
| *** Hefeweizen has joined #openstack-infra | 20:38 | |
| *** dtroyer has quit IRC | 20:38 | |
| *** dtroyer has joined #openstack-infra | 20:38 | |
| *** annegentle has joined #openstack-infra | 20:39 | |
| *** gyee has quit IRC | 20:42 | |
| *** bknudson has left #openstack-infra | 20:42 | |
| *** derekh has joined #openstack-infra | 20:45 | |
| *** sarob has joined #openstack-infra | 20:47 | |
| *** hogepodge has quit IRC | 20:50 | |
| *** hogepodge has joined #openstack-infra | 20:51 | |
| *** mrda has joined #openstack-infra | 20:52 | |
| *** mrda has quit IRC | 20:55 | |
| *** ^d has joined #openstack-infra | 20:55 | |
| *** CaptTofu has quit IRC | 20:56 | |
| *** CaptTofu has joined #openstack-infra | 20:56 | |
| *** CaptTofu has quit IRC | 21:01 | |
| clarkb | fungi: I am back | 21:01 |
| fungi | clarkb: for n in $( sudo nodepool list | grep delete | cut -d' ' -f2 ) ; do sudo nodepool delete $n ; done | 21:01 |
| fungi | that's what i did last time | 21:01 |
| clarkb | Mithrandir: yup I got that sorted. A friend had gone to Australia recently and didn't do it and did it in the airport prior to take off and was fine | 21:02 |
| clarkb | fungi: thanks | 21:02 |
| clarkb | fungi: that list is 77 long | 21:03 |
| clarkb | which seems to be about right for the purple bits of the graph | 21:03 |
| fungi | yup | 21:03 |
| fungi | you could split the list and run 5 or so in parallel if you want. the latency on api responses from nova will make it take a while to finish otherwise | 21:04 |
| clarkb | meh I will just run that in the background and be patient (I have other things to do while I wait) | 21:04 |
| fungi | that was my take on it when i last ran it | 21:05 |
| clarkb | it is running now | 21:05 |
| *** xeyed4good has joined #openstack-infra | 21:05 | |
| clarkb | I am going to propose a change to pull out my ssh key and will push that through if I can get you to +2 it | 21:05 |
| fungi | yup, same from me in a bit here | 21:06 |
| *** blamar has quit IRC | 21:07 | |
| *** ericw has joined #openstack-infra | 21:07 | |
| *** blamar has joined #openstack-infra | 21:07 | |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Revert "Add temporary summit pubkey for clarkb." https://review.openstack.org/56094 | 21:08 |
| clarkb | fungi: ^ | 21:08 |
| *** julim has quit IRC | 21:11 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/jeepyb: Remove run-mirror from jeepyb https://review.openstack.org/56096 | 21:11 |
| mordred | clarkb: ^^ there's the change to remove run-mirror from jeepyb, now that pypi-mirror is there | 21:11 |
| clarkb | mordred: +2'd | 21:12 |
| clarkb | fungi: I will replace my old key in a bit when I have time to deal with that, but for now figure removing the temporary key is prudent | 21:12 |
| clarkb | mordred: I should probably wait for jenkisn to +1 but it lgtm :) | 21:13 |
| *** svarnau has quit IRC | 21:13 | |
| *** davidhadas has quit IRC | 21:14 | |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Remove fungi's temporary summit and old 2012 keys https://review.openstack.org/56097 | 21:14 |
| *** jsergent has quit IRC | 21:14 | |
| *** davidhadas has joined #openstack-infra | 21:15 | |
| *** senk has joined #openstack-infra | 21:16 | |
| openstackgerrit | A change was merged to openstack-infra/config: Revert "Add temporary summit pubkey for clarkb." https://review.openstack.org/56094 | 21:16 |
| openstackgerrit | A change was merged to openstack-infra/config: Remove fungi's temporary summit and old 2012 keys https://review.openstack.org/56097 | 21:18 |
| *** toddmorey has joined #openstack-infra | 21:19 | |
| *** denis_makogon has quit IRC | 21:19 | |
| *** sarob has quit IRC | 21:19 | |
| *** sarob has joined #openstack-infra | 21:20 | |
| *** dprince has quit IRC | 21:21 | |
| openstackgerrit | Russell Bryant proposed a change to openstack-infra/reviewstats: Add received reviews to review stats https://review.openstack.org/55999 | 21:22 |
| *** vipul is now known as vipul-away | 21:24 | |
| *** sarob has quit IRC | 21:24 | |
| *** jsergent has joined #openstack-infra | 21:26 | |
| *** sarob has joined #openstack-infra | 21:26 | |
| openstackgerrit | Russell Bryant proposed a change to openstack-infra/reviewstats: Add received reviews to review stats https://review.openstack.org/55999 | 21:26 |
| *** xeyed4good has quit IRC | 21:26 | |
| *** markmc has quit IRC | 21:27 | |
| devananda | mordred: I didn't make it to the storyboard session. is there a summary somewhere? should we continue using BPs? | 21:28 |
| mordred | devananda: yes | 21:29 |
| mordred | devananda: storyboard is not ready for you yet | 21:29 |
| devananda | ack | 21:29 |
| openstackgerrit | Russell Bryant proposed a change to openstack-infra/reviewstats: Add received reviews to review stats https://review.openstack.org/55999 | 21:30 |
| jamesmcarthur | mordred: FYI - Member Profile affiliations have been fixed. | 21:30 |
| mordred | jamesmcarthur: thanks! | 21:30 |
| jamesmcarthur | NP - I'll work with Todd/Stefano on getting word out to the community | 21:31 |
| jamesmcarthur | mordred: but we didn't have perfect data to work with on this feature | 21:31 |
| mordred | russellb: oh no. your new patches mean you're going to see my patch to review ration!!! | 21:31 |
| *** vipul-away is now known as vipul | 21:31 | |
| *** blamar has quit IRC | 21:31 | |
| russellb | mordred: that's the idea | 21:31 |
| mordred | russellb: NOOOOOO | 21:31 |
| russellb | mordred: still toying with how to show it exactly | 21:31 |
| russellb | right now i'm showing reviews given vs reviews received | 21:31 |
| clarkb | review ration? | 21:31 |
| *** blamar has joined #openstack-infra | 21:32 | |
| *** pblaho has quit IRC | 21:32 | |
| mordred | russellb: I'd been trying to get my damned percentage down already, now you're going to give me a new thing to improve myself with? | 21:32 |
| russellb | http://paste.openstack.org/show/52163/ | 21:32 |
| mordred | clarkb: raio | 21:32 |
| mordred | gah | 21:32 |
| mordred | ratio | 21:32 |
| clarkb | oh ratio | 21:32 |
| jamesmcarthur | mordred: so there are some cases where the Org affiliation may say current, where it is not, etc… but the idea here is to begin to put a timeline of Organization Affiliation for all OpenStack Members | 21:32 |
| *** hogepodge has quit IRC | 21:33 | |
| mordred | jamesmcarthur: yes. I like this very much. | 21:33 |
| *** nati_ueno has joined #openstack-infra | 21:34 | |
| russellb | infra version - http://paste.openstack.org/show/52164/ | 21:34 |
| *** hogepodge has joined #openstack-infra | 21:34 | |
| russellb | mordred: you have a good ratio :) | 21:35 |
| mordred | russellb: oh thank goodness | 21:35 |
| jamesmcarthur | mordred: I'll try to make sure I stay on this channel over the next few days in the event anyone has any concerns/problems, but QA would seem to indicate the deployment/script execution went well | 21:35 |
| *** jhesketh__ has quit IRC | 21:36 | |
| mordred | jamesmcarthur: you should just hang out here in general! :) | 21:36 |
| anteaya | bah I'm only half of waht pleia2 is doing, I need to get back to reviewing again | 21:36 |
| jamesmcarthur | mordred: I lurk occassionally ;) | 21:36 |
| *** hogepodge has quit IRC | 21:37 | |
| *** smarcet has left #openstack-infra | 21:37 | |
| *** gyee has joined #openstack-infra | 21:41 | |
| *** blamar has quit IRC | 21:42 | |
| mikal | How do I found out what happened with elastic recheck with https://review.openstack.org/#/c/55492/ ? | 21:42 |
| mikal | It says it did a recheck last night, but it hasn't actually posted results | 21:43 |
| mikal | Perhaps it failed to actually trigger the recheck | 21:43 |
| mikal | ? | 21:43 |
| clarkb | mikal: it doesn't actually do a recheck | 21:43 |
| mikal | Ahhh, ok | 21:43 |
| mikal | I shall manually do one then | 21:43 |
| clarkb | mikal: it is providing you with enough info to make a manual recheck easy | 21:43 |
| fungi | that is still only a future possibility, once we're all more comfortable with its accuracy i think | 21:43 |
| clarkb | mikal: the idea being that having a human double check the automated results is probably a good thing | 21:43 |
| clarkb | fungi: right | 21:43 |
| mikal | Oh, that wasn't clear to me | 21:44 |
| mikal | I thought it was doing magic | 21:44 |
| mikal | Bummer man | 21:44 |
| fungi | but i agree, sticking the dev in that pipeline is a good thing, at least for now | 21:44 |
| fungi | mikal: it IS doing SO MUCH magic | 21:44 |
| mikal | NEED MORE MAGIC | 21:44 |
| *** DennyZhang has joined #openstack-infra | 21:44 | |
| fungi | are you sure? it's a but runny | 21:44 |
| fungi | er, a bit | 21:45 |
| lifeless | I thought you were putting on an accent then :) | 21:45 |
| *** CaptTofu has joined #openstack-infra | 21:45 | |
| * fungi wears as few accents as he can manage | 21:45 | |
| mikal | Heh | 21:46 |
| fungi | a bít (now *there's* an accent) | 21:46 |
| *** jhesketh_ has joined #openstack-infra | 21:46 | |
| *** jhesketh__ has joined #openstack-infra | 21:46 | |
| *** CaptTofu has quit IRC | 21:47 | |
| *** blamar has joined #openstack-infra | 21:47 | |
| *** CaptTofu has joined #openstack-infra | 21:48 | |
| *** CaptTofu has quit IRC | 21:49 | |
| *** vipul is now known as vipul-away | 21:49 | |
| *** vipul-away is now known as vipul | 21:49 | |
| *** CaptTofu has joined #openstack-infra | 21:50 | |
| *** derekh has quit IRC | 21:51 | |
| *** jsergent has quit IRC | 21:52 | |
| *** pcm_ has quit IRC | 21:53 | |
| *** nati_ueno has quit IRC | 21:55 | |
| *** nati_ueno has joined #openstack-infra | 21:55 | |
| openstackgerrit | Arx Cruz proposed a change to openstack-infra/devstack-gate: Adding an option to use qpid instead of rabbit or zeromq https://review.openstack.org/55829 | 21:57 |
| pabelanger | afternoon! So, to continue with my packaging proposal. What do people think of having debian/sid, debian/jessey, debian/wheezy, ubuntu/precise, etc for branch names in the zuul-packaging repo | 21:57 |
| ArxCruz | lifeless ^ | 21:57 |
| ArxCruz | hope you like :) | 21:58 |
| *** fifieldt has joined #openstack-infra | 21:58 | |
| ArxCruz | mordred: fungi what should I do in this case https://review.openstack.org/#/c/55827/ ? | 21:58 |
| ArxCruz | clarkb: ^ | 21:58 |
| pabelanger | or do we want debian/unstable, debian/stable, debian/experimental, etc | 21:59 |
| mordred | ArxCruz: you wanted "recheck bug XXXXX" not reverify | 21:59 |
| ArxCruz | mordred: i didn't get it ... :/ | 21:59 |
| mordred | pabelanger: I like teh code names | 21:59 |
| mordred | ArxCruz: there are two trigger verbs - recheck and reverify | 21:59 |
| clarkb | pabelanger: I am a fan of not needing to rename those branches each time the distro shifts, but code names would make for more consistency with fedora and ubuntu | 22:00 |
| *** nati_ueno has quit IRC | 22:00 | |
| mordred | clarkb: you don't need to rename - you need to make a new branch when a new release is made | 22:00 |
| mordred | the old names rename immutably what they are for all time | 22:00 |
| ArxCruz | mordred: oh, cool :) | 22:00 |
| mordred | ArxCruz: recheck is to retrigger check jobs - reverify is to retrigger gate jobs | 22:01 |
| ArxCruz | mordred: cool, thanks :) | 22:01 |
| pabelanger | mordred, clarkb okay, so I'll move master to debian/sid, then add some documentation into master about the branches. Give that a test, and see what would be needed to import into openstack-infra | 22:01 |
| ArxCruz | fungi: by the way, can I have you +2 in https://review.openstack.org/#/c/54196/ ? :D | 22:02 |
| mordred | pabelanger: have you given any thought to using the git-buildpackage repo to automatically build packages from trunk? (I have some thoughts - but curious if you've thought of that usecase yet) | 22:02 |
| pabelanger | mordred, Yes, I have something going today using pbuilder / chroots to build some asterisk packages I use. It works out pretty well however I'd be interested in hearing other ways to do it | 22:04 |
| mordred | pabelanger: awesome | 22:04 |
| *** jsergent has joined #openstack-infra | 22:05 | |
| mordred | my main thing is that I'd love to have, you know, one job that we can trigger for anything that has both a code repo and a packaging repo | 22:05 |
| pabelanger | Ya, that would be hotness | 22:05 |
| pabelanger | I had gates working with pbuilder too. The setup is pretty static, but I don't think it would take much to import the stuff into puppet and then nodepool | 22:07 |
| *** hogepodge has joined #openstack-infra | 22:07 | |
| *** syerrapragada has quit IRC | 22:09 | |
| *** syerrapragada has joined #openstack-infra | 22:10 | |
| *** hashar has quit IRC | 22:12 | |
| *** nati_ueno has joined #openstack-infra | 22:13 | |
| fungi | ArxCruz: looking | 22:16 |
| ArxCruz | fungi: you're da man! :D | 22:16 |
| fungi | mordred: pabelanger: main thing i'd worry about with something like pbuilder is making sure we keep those chroots updated | 22:16 |
| fungi | assuming we cache them at all | 22:17 |
| fungi | but a full debootstrap run for every build would be kinda slow, so we probably should | 22:17 |
| mordred | fungi: I believe we have some infrastructure thoughts around keeping base images up to date :) | 22:18 |
| *** dkranz has quit IRC | 22:18 | |
| fungi | yup | 22:19 |
| fungi | i figure we'd do something similar | 22:19 |
| pabelanger | fungi, Agreed. For my uses, I had a two step process. Night crontabs to pbuilder update, then hooks before each build to update, dist-upgrade. It worked well for my needs, however it required a lot of caching and disk space. | 22:19 |
| *** jorisroovers has quit IRC | 22:19 | |
| fungi | yeah, if we ran these on devstack slaves we could just build the chroots during the slave snapshot creation each day | 22:20 |
| *** changbl has quit IRC | 22:20 | |
| fungi | alternatively we could rely on jenkins slaves to cache the tarball in their workspace and simply update it as a first step in the job, creating a new one if nonexistent | 22:21 |
| openstackgerrit | lifeless proposed a change to openstack-infra/reviewstats: Add slage as TripleO core. https://review.openstack.org/56103 | 22:21 |
| *** hogepodge has quit IRC | 22:21 | |
| *** gyee has quit IRC | 22:22 | |
| *** nati_ueno has quit IRC | 22:24 | |
| *** dstanek has quit IRC | 22:24 | |
| *** nati_ueno has joined #openstack-infra | 22:24 | |
| openstackgerrit | lifeless proposed a change to openstack-infra/reviewstats: Add TOCI as a TripleO review responsibility. https://review.openstack.org/56104 | 22:24 |
| *** dstanek has joined #openstack-infra | 22:27 | |
| *** gyee has joined #openstack-infra | 22:28 | |
| *** hashar has joined #openstack-infra | 22:28 | |
| *** che-arne has joined #openstack-infra | 22:28 | |
| *** DennyZhang has quit IRC | 22:29 | |
| pabelanger | Just to confirm, openstack-infra/zuul-packaging right? VS stackforge | 22:29 |
| *** weshay has quit IRC | 22:29 | |
| *** nati_uen_ has joined #openstack-infra | 22:29 | |
| *** nati_ueno has quit IRC | 22:29 | |
| *** mfer has quit IRC | 22:30 | |
| fungi | pabelanger: yep | 22:30 |
| clarkb | ++ | 22:31 |
| *** dkliban has quit IRC | 22:32 | |
| *** nati_uen_ has quit IRC | 22:33 | |
| *** nati_ueno has joined #openstack-infra | 22:34 | |
| clarkb | fungi: the nodepool extra nodes count has stayed low, I am about to do a threaddump | 22:34 |
| clarkb | should I go ahead and restart nodepool? | 22:34 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/zuul: Make merger.Repo more context manager like. https://review.openstack.org/55943 | 22:35 |
| clarkb | fungi: ^ is ready for review (though it depends on a change that needs better testing | 22:35 |
| *** oubiwann has quit IRC | 22:35 | |
| fungi | clarkb: yeah, can't hurt | 22:35 |
| clarkb | ok doing threaddump now | 22:35 |
| *** julim has joined #openstack-infra | 22:36 | |
| *** nati_ueno has quit IRC | 22:38 | |
| clarkb | fungi: threaddump is in my homedir (so that we don't have to hunt it down in logs later), I am restarting the service now | 22:38 |
| fungi | k | 22:39 |
| *** toddmorey has quit IRC | 22:40 | |
| clarkb | "waiting ......" I should remember to do this stuff in screen or something more reliable that plain ssh | 22:40 |
| clarkb | I am assuming it is waiting for image and or server builds requests to complete | 22:40 |
| fungi | yeah | 22:41 |
| fungi | it doesn't usually take too long unless there's an image build happening | 22:41 |
| ArxCruz | fungi: ty for the review | 22:44 |
| fungi | ArxCruz: yw. as i mentioned there i'm hesitant to approve non-emergency changes to zuul while jeblair is on vacation, but it looks great and i agree making that configurable is the right solution | 22:45 |
| ArxCruz | fungi: :) I already wait a week, one more week will not hurt :) | 22:46 |
| clarkb | ya I have a couple zuul changes up too but don't expect to merge them until jeblair gets a chance to review them | 22:46 |
| clarkb | still waiting on nodepool. I am an impatient person | 22:46 |
| openstackgerrit | A change was merged to openstack-infra/reviewstats: Add slage as TripleO core. https://review.openstack.org/56103 | 22:46 |
| *** jpich has quit IRC | 22:47 | |
| clarkb | fungi: have you done a nodepoold restart before? I am not entirely convinced that the do_graceful_stop bits of the init script work properly | 22:52 |
| clarkb | fungi: because the various threads running in nodepool don't seem to be daemon threads | 22:53 |
| *** toddmorey has joined #openstack-infra | 22:53 | |
| openstackgerrit | Paul Belanger proposed a change to openstack-infra/config: Import zuul-packaging into openstack-infra https://review.openstack.org/56107 | 22:53 |
| clarkb | fungi: I think I am going to kill it more forcefully unless you have different info | 22:53 |
| zaro | clarkb: japan air has USB at every seat. | 22:53 |
| clarkb | zaro: air canada had it too. they even had 110v in a bunch of seats | 22:54 |
| clarkb | but 110 wasn't in every seat | 22:54 |
| sdague | zaro: so did cathay pacific | 22:54 |
| zaro | seriously? which ones? | 22:54 |
| zaro | i was pleasantly surprised by that. United flight did not | 22:54 |
| clarkb | zaro: I think middle and window seat had 110 | 22:55 |
| clarkb | I was in the aisle so didn't have it | 22:55 |
| clarkb | fungi: I am going to give it to 2300 UTC and kill it more forcefully if it doesn't restart gracefully | 22:56 |
| *** senk has quit IRC | 22:59 | |
| fungi | cool | 23:00 |
| clarkb | ok kicking it now | 23:00 |
| clarkb | stop then start seems to have worked | 23:00 |
| openstackgerrit | A change was merged to openstack-infra/reviewstats: Add some core team review stats https://review.openstack.org/55995 | 23:02 |
| *** zehicle_at_dell has quit IRC | 23:06 | |
| *** thomasem has quit IRC | 23:08 | |
| *** dcramer_ has quit IRC | 23:14 | |
| *** dstanek has quit IRC | 23:14 | |
| *** masayukig has joined #openstack-infra | 23:17 | |
| masayukig | jog0: ping | 23:20 |
| jog0 | masayukig: pong | 23:20 |
| masayukig | about https://review.openstack.org/#/c/55982/ | 23:21 |
| masayukig | I don't understand why elastic recheck can't catch the bug 1225664. | 23:21 |
| masayukig | Patch Set 17 of https://review.openstack.org/#/c/49892/ . | 23:22 |
| *** hashar has quit IRC | 23:23 | |
| masayukig | May the elastic recheck miss the query? | 23:23 |
| *** jhesketh_ has quit IRC | 23:23 | |
| *** dims has quit IRC | 23:23 | |
| jog0 | masayukig: looking | 23:24 |
| masayukig | thanks | 23:24 |
| masayukig | jog0: thanks. | 23:24 |
| *** mriedem has quit IRC | 23:24 | |
| *** dstanek has joined #openstack-infra | 23:26 | |
| jog0 | masayukig: strange, clarkb ^^ any ideas | 23:27 |
| jog0 | http://logstash.openstack.org/#eyJmaWVsZHMiOltdLCJzZWFyY2giOiJtZXNzYWdlOlwiTm8gc3BhY2UgbGVmdCBvbiBkZXZpY2VcIiBBTkQgZmlsZW5hbWU6XCJsb2dzL3N5c2xvZy50eHRcIiBBTkQgc3lzbG9nX3Byb2dyYW06XCJvYmplY3Qtc2VydmVyXCIgQU5EIGJ1aWxkX2NoYW5nZTpcIjQ5ODkyXCIiLCJ0aW1lZnJhbWUiOiI2MDQ4MDAiLCJncmFwaG1vZGUiOiJjb3VudCIsIm9mZnNldCI6MCwidGltZSI6eyJ1c2VyX2ludGVydmFsIjowfSwibW9kZSI6IiIsImFuYWx5emVfZmllbGQiOiIiLCJzdGFtcCI6MTM4NDI5ODc4NDYwOH0= | 23:27 |
| jog0 | shows there should be a hit | 23:27 |
| *** ^d has quit IRC | 23:27 | |
| jog0 | clarkb: perhaps there is an issue with syslog parsing, we may not wait long enough for that file to get loaded | 23:28 |
| clarkb | jog0: filename:"logs/syslog.txt" AND message:"No space left on device" AND syslog_program:"object-server" AND build_uuid:c416c79* returns one result in the last 24 hours which is what we epxected | 23:29 |
| clarkb | jog0: and looking at the timestamps we indexed that line about 30 seconds after the test finished | 23:30 |
| clarkb | so ya, I think maybe bug in e-r? | 23:30 |
| jog0 | hrm perhaps | 23:33 |
| *** julim has quit IRC | 23:33 | |
| *** jamesmcarthur has quit IRC | 23:34 | |
| jog0 | running it locally to see | 23:35 |
| clarkb | or maybe, your change to update *5664 didn't get applied until later? /me checks process start time | 23:35 |
| clarkb | ps says it has been running since 02:20 which was hours before that test finished | 23:36 |
| jog0 | clarkb: we *should* be reading the file each time a new failure comes in | 23:36 |
| clarkb | ya | 23:36 |
| *** dims has joined #openstack-infra | 23:37 | |
| clarkb | fungi: https://review.openstack.org/#/c/55943/ can you see my comment there and leave a response if you have something better in mind | 23:39 |
| clarkb | jog0: perhaps we need to be restricting the build uuid on every query | 23:40 |
| clarkb | rather than at the end as a sanity check | 23:40 |
| clarkb | jog0: I wonder if it is possible for results to come back but non of the build uuids match because we have enough results | 23:40 |
| clarkb | *more than enough results leading to e-r dropping that particular thing | 23:41 |
| *** michchap has quit IRC | 23:44 | |
| fungi | clarkb: i'm thinking there's a way to count the number of instances of a class in the global context... i'll see if i can jog my recollection | 23:44 |
| *** datsun180b has quit IRC | 23:45 | |
| *** michchap has joined #openstack-infra | 23:47 | |
| clarkb | there is a get_referrers, I wonder if that works on class objects and not instances | 23:47 |
| jog0 | clarkb: hmm that is a good guess | 23:48 |
| clarkb | fungi: it looks like get_referrers may work with some magic | 23:49 |
| *** slong has quit IRC | 23:51 | |
| clarkb | fungi: the problem I am having is you need to keep a reference to check that the reference doesn't exist which means the reference will exist | 23:52 |
| clarkb | fungi: there may be a way to look at it sideways and see if anything of the sort doesn't actually exist | 23:53 |
| jog0 | clarkb: hmm bug 5664 is sometimes classified, which fits your theory | 23:53 |
| clarkb | jog0: it happens multiple times per job it looks like. so that would bump up the count and since we only look at the first 10 results we may be confusing ourselves | 23:53 |
| clarkb | jog0: in any case being more specific can't hurt | 23:54 |
| jog0 | clarkb: agreed, want to push out a fix? | 23:54 |
| *** jergerber has quit IRC | 23:55 | |
| clarkb | jog0: I am currently working on a fix for a zuul problem | 23:55 |
| clarkb | I can probably poke at e-r tomorrow but not today | 23:55 |
| fungi | clarkb: i have something with get_referrers i think, since you can check its dict for a string key matching the instance name | 23:56 |
| fungi | lemme mock up a confirmation and i'll add it in a comment | 23:57 |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/elastic-recheck: Remove fixed bugs from queries.yaml https://review.openstack.org/55914 | 23:57 |
| fungi | it does make somewhat deep assumptions about the gc implementation though, so might not be considered a clean test | 23:57 |
| *** dstanek has quit IRC | 23:57 | |
| jog0 | clarkb: that works for me, thanks | 23:58 |
| jog0 | masayukig: thanks for bringing this to our attention | 23:58 |
| masayukig | jog0: you are welcome :) | 23:59 |
| clarkb | fungi: ya, that is what I am leaning towards, I am running something like you describe in the zuul test suite now | 23:59 |
| clarkb | fungi: curious to see what you come up with | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!