Friday, 2018-04-20

clarkbI'm not seeing a source fs option for mkfs.xfs which may be why00:00
clarkbthis is likely why they went the route of loopback mounting00:01
clarkbit is ultimately more flexible but does cost you root00:01
tobiashProbably00:04
*** rlandy has quit IRC00:46
*** harlowja has quit IRC00:46
openstackgerritJames E. Blair proposed openstack-infra/zuul master: Late bind projects  https://review.openstack.org/55361801:20
openstackgerritJames E. Blair proposed openstack-infra/zuul master: WIP: Make config objects freezable  https://review.openstack.org/56281601:20
*** threestrands has joined #zuul02:59
*** swest has joined #zuul05:07
*** elyezer has quit IRC06:29
*** hashar has joined #zuul06:31
openstackgerritMarkus Hosch proposed openstack-infra/zuul master: Store build start/end time in UTC  https://review.openstack.org/56227306:49
*** threestrands has quit IRC07:00
openstackgerritBenedikt Löffler proposed openstack-infra/zuul master: Add role information to task in zuul_json callback  https://review.openstack.org/55999807:10
openstackgerritBenedikt Löffler proposed openstack-infra/zuul master: Add role information to task in zuul_json callback  https://review.openstack.org/55999807:34
*** ssbarnea_ has joined #zuul08:00
*** jpena|off is now known as jpena08:00
*** electrofelix has joined #zuul08:33
*** hashar has quit IRC09:19
*** bhavik1 has joined #zuul09:39
*** rcarrillocruz has joined #zuul09:40
*** bhavik1 has quit IRC09:47
*** gundalow has joined #zuul10:04
*** xinliang has quit IRC10:42
*** xinliang has joined #zuul10:42
*** xinliang has quit IRC10:42
*** xinliang has joined #zuul10:42
*** pwhalen has quit IRC10:43
*** pwhalen has joined #zuul10:44
*** pwhalen has joined #zuul10:44
*** ssbarnea_ has quit IRC11:00
*** ssbarnea_ has joined #zuul11:01
*** ssbarnea_ has quit IRC11:11
*** ssbarnea_ has joined #zuul11:14
*** rlandy has joined #zuul12:32
*** elyezer has joined #zuul12:40
*** JasonCL has quit IRC13:35
tobiashcorvus: I have a use case I currently don't know how to solve this: a project wants a buildset which first builds a docker image (if some files changed) and after that several jobs that use that image or the current latest image13:40
tobiashusing job dependencies doesn't work because the child jobs won't run if the first is filtered out13:40
tobiashdo you think it would be an option to mark dependencies to non-existing jobs in the current buildset satisfied?13:42
tobiashI think that could make that optional dependency work13:42
openstackgerritDavid Shrewsbury proposed openstack-infra/zuul master: WIP: Update to Ansible 2.5  https://review.openstack.org/56266813:47
corvustobiash: you could always run the build job and have it short-circuit if nothing has changed13:47
tobiashcorvus: yes, that was my first thought, but that still would allocate an unneeded node, push repos there and then notice that nothing has to be done13:48
Shrewsthis is getting fun: https://review.openstack.org/#/c/562668/5/zuul/ansible/callback/zuul_json.py13:48
tobiashso that is not an optimal solution13:48
tobiashShrews: oh, a try-catch chain13:49
corvustobiash: true.  i think it'd have to be an option then, that you declare on the dependency.  because sometimes you may want that behavior, or sometimes you may want the behavior where nothing runs.13:49
tobiashcorvus: as dependencies is currently just a list, add a weak-dependencies?13:50
*** JasonCL has joined #zuul13:50
tobiashcorvus: two variants: https://etherpad.openstack.org/p/ORaD48jE5613:52
tobiashI think I'd favor the first one13:53
*** dims has quit IRC13:58
corvustobiash: yeah, we've done the string-or-dict thing before.  i prefer it to option 2.  there's also option 3.14:00
*** mnaser has quit IRC14:00
*** mnaser has joined #zuul14:01
tobiashcorvus: option 2 would be fine for me either14:01
tobiashthat's probably the simplest for the user14:02
*** dims has joined #zuul14:04
Shrewsw00t. got 2.5 working. well, 2.5.114:13
Shrewsi need to set that as the min in requirements.txt i guess since we didn't actually test 2.5.014:14
openstackgerritDavid Shrewsbury proposed openstack-infra/zuul master: Update to Ansible 2.5  https://review.openstack.org/56266814:16
ShrewsunWIP'd14:17
corvusreminder: today would be a great today to jot down some notes in the weekly update etherpad: https://etherpad.openstack.org/p/zuul-update-email14:21
*** acozine1 has joined #zuul14:31
*** nguyenhai has quit IRC14:33
*** JasonCL has quit IRC14:43
*** JasonCL has joined #zuul14:45
tobiashShrews: yay, thanks for doing my task for this week :)14:45
corvuswe should say tobiash plans to cheer Shrews on about upgrading ansible :)14:49
Shrewsthat's what i do. hide behind the scenes making others look good  :)14:51
Shrewsthis is how mordred became so successful14:51
Shrews:D14:52
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Add regex support to project stanzas  https://review.openstack.org/53571315:06
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Add regex support to project stanzas  https://review.openstack.org/53571315:08
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Add regex support to project stanzas  https://review.openstack.org/53571315:30
tobiashcorvus: that conflicted with your config refactor stack so I rebased it onto your stack ^15:32
*** electrofelix has quit IRC15:33
openstackgerritMerged openstack-infra/zuul master: Update encrypt_secret to be API aware  https://review.openstack.org/56168115:40
*** rbergeron has quit IRC15:49
*** dims has quit IRC16:06
*** dims has joined #zuul16:11
SpamapShas anybody made a role for pulling all the journals on a box with journalctl?16:30
pabelangerSpamapS: not yet, I've just restored to in-tree playbook: http://git.openstack.org/cgit/openstack/windmill/tree/tests/collect-logs.yaml But would happily use a role if there was one16:32
pabelangerresorted*16:33
clarkbSpamapS: pabelanger devstack does something like that16:38
clarkbmaybe that role should be made more generic16:39
clarkbWe do human readable output for openstack services and "syslog" but also dump the full journal in an export format then xz them (as it does far better than gzip on the data)16:39
pabelangerclarkb: yah, making it more generic would be helpful16:40
clarkbSpamapS: are you looking for the export format or human readable or both?16:42
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Only emit parent-change-enqueued if needed  https://review.openstack.org/56319416:50
*** rlandy is now known as rlandy|brb16:52
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Limit search scope of getChangesDependingOn to tenant  https://review.openstack.org/56319617:09
*** jpena is now known as jpena|off17:09
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Only emit parent-change-enqueued if needed  https://review.openstack.org/56319417:10
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Limit search scope of getChangesDependingOn to tenant  https://review.openstack.org/56319617:10
corvusclarkb, tobiash: copy.deepcopy doesn't work on a mappingproxy applied to an ordereddict :(17:12
clarkbcorvus: it isn't a recursive copy?17:12
clarkbI guess that makes sense? unfortunate though17:12
corvusyeah, i think it's trying to reconstruct a mappingproxy17:13
corvusoh, actually, it may not even work on a mappingproxy with a regular dict17:13
corvusyep, this fails: copy.deepcopy(types.MappingProxyType({}))17:14
*** rlandy|brb is now known as rlandy17:14
tobiashcorvus: do we still have to do deep copies with your refactor?17:14
corvusthat puts a crimp in my plans17:14
corvustobiash: i'm not sure -- the main thing i'm worried about is when we create frozen jobs from variants.  we copy a bunch of the variant attributes onto the job.  it's possible some or all of those don't need to be deepcopies anymore.17:15
corvus(indeed, "# TODO(jeblair): determine if deepcopy is required" it already may not be required :)17:16
corvusi guess i will be doing that todo right now :)17:18
tobiash:)17:18
tobiashand yay finally I have at least one item on my list from this week17:19
corvuswow i found an undocument/unused job attribute :|17:20
corvusokay, that section is safe to remove deepcopy.  i think i also need to check variables.17:23
corvusthis is going to take some time, but i think it's going to be worth doing (as it should clean up a bunch of unecessary copying)17:23
openstackgerritJames E. Blair proposed openstack-infra/zuul master: WIP: Make config objects freezable  https://review.openstack.org/56281617:24
corvusthat's just a checkpoint, i have to afk for a bit17:24
AJaegercorvus, tobiash, tristanC, regarding https://review.openstack.org/548082 - should we handle a single command - or allow multiple ones as tristanC suggests?17:25
tobiashI think if we do that job I'd slightly prefer a command list as that's more flexible for the easy use cases17:27
tobiashbut a downside of this is that it encourages overuse17:27
openstackgerritTobias Henkel proposed openstack-infra/zuul master: DNM: test tox-cover  https://review.openstack.org/56319917:33
AJaegerlet me try that in a separate change...17:39
tobiashAJaeger: commented on 55934817:42
AJaegertobiash: thanks17:48
openstackgerritAndreas Jaeger proposed openstack-infra/zuul-jobs master: Add simple run-test-commands job  https://review.openstack.org/56320417:52
AJaegertristanC: that's the alternative ^17:53
fungii'm not sure what the reason is for the +2 dog-piling on 535543 but i'm going to break the cycle by approving it18:05
fungior does tristanC need to acknowledge the patchset?18:05
fungisince i guess he wasn't the uploader for ps1718:06
SpamapSclarkb: I have roles that pull individual journals in human readable format for convenience and debugging.18:06
SpamapSclarkb: I wouldn't know what to do with the binary versions. :-P18:07
SpamapSWould be really cool if I could do the binary versions and then html-prettied versions.18:07
fungii think the devstack docs mention how to make use of the exported binary format18:08
fungiarguably convenient since you get things like inherent filtering support18:09
fungibut yes, i always end up having to reread instructions for that18:09
SpamapSYeah if I knew the tools I'd probably like that. Though usually I just want to poke around in the browser and find a string or two18:09
SpamapSlike "what sploded?"18:09
fungisame here18:13
clarkbthe killer feature is having everything interleaved but being able to filter out what you don't want18:23
clarkbrather than multiple disparate human readable files18:23
*** hashar has joined #zuul18:25
openstackgerritAndreas Jaeger proposed openstack-infra/zuul-jobs master: Add simple run-test-commands job  https://review.openstack.org/56320418:35
fungihttps://review.openstack.org/563217 is a straw man for the sb project group reorganizing i mentioned on tuesday18:36
fungijust took a stab at guessing which repos we want in which group(s)18:37
AJaegertristanC: could you check https://review.openstack.org/#/c/563204 , please? That's an alternative18:46
* AJaeger signes off and wishes a great weekend18:46
fungihave a great weekend yourself, AJaeger!18:46
AJaegerthanks!18:46
*** harlowja has joined #zuul19:02
openstackgerritTobias Henkel proposed openstack-infra/zuul master: WIP: Test parent-change-enqueued with github  https://review.openstack.org/56324219:11
*** acozine1 has quit IRC19:11
*** JasonCL has quit IRC19:13
*** elyezer has quit IRC19:14
SpamapSwhat I really want is ARA but for journals19:18
SpamapSso like "show me only errors"19:18
openstackgerritmelissaml proposed openstack-infra/zuul master: Trivial: Update pypi url to new url  https://review.openstack.org/56324819:19
*** elyezer has joined #zuul19:26
*** JasonCL has joined #zuul20:33
*** hashar has quit IRC20:39
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Test parent-change-enqueued with github  https://review.openstack.org/56324220:47
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Extend github testing using app auth  https://review.openstack.org/56329720:47
*** ssbarnea_ has quit IRC21:04
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Support getting the github client via installation id  https://review.openstack.org/56331121:06
corvusSpamapS: maybe os-21:20
corvusgr21:21
corvusSpamapS: maybe os-loganalyze can be expanded to support journal files, if it doesn't already?21:21
SpamapScorvus: yeah that would be sweet21:21
corvusclarkb: ^21:21
corvusmy feeling is that it's not *very* openstack specific, so could probably be generalized without too much trouble21:22
clarkbit came up when we added those files to the log collection21:22
corvusi thought that sounded familiar :)21:22
clarkbthe format is quite good for machines. My only concern with a direct rendering is I'm unsure of how stable the format is21:23
clarkbcan also use journalctl and similar tools to deal with it if not very stable21:23
corvusi'm going to bow out of this conversation though, cause i don't have anything nice to say about binary logfiles21:23
clarkboh the export format isn't binary21:25
clarkbat least I don't think it is21:25
SpamapSI'm pretty sure journalctl is the only safe way.21:35
SpamapSunless it has an underlying library.21:35
*** rlandy has quit IRC21:53
SpamapSneed second set of eyes22:32
SpamapShttp://paste.openstack.org/show/719669/22:32
SpamapSwhat's wrong with that?22:32
SpamapShttp://paste.openstack.org/show/719670/ <-- error I'm getting22:33
clarkbcorvus: https://review.openstack.org/#/c/562273/3 straightforward timestamp + database + timezone issue if you want to take a look (I didn't approve because I think eyeballs on potentially end user behavior alterations even if minor deserve eyeballs)22:35
clarkbSpamapS: my reading of that is zuul expects a single string dependencies value?22:37
*** JasonCL has quit IRC22:37
clarkbrather than a list type22:37
clarkbthe docs definitely say it should be a list though22:38
*** JasonCL has joined #zuul22:38
*** JasonCL has joined #zuul22:39
clarkbSpamapS: http://paste.openstack.org/show/719671/ is an example from the tests, maybe try converting your jobs: list to be like that rather than name: foo\ndependencies:22:40
clarkbSpamapS: but keep it as a list22:40
openstackgerritMerged openstack-infra/zuul master: Remove extra words from gerrit setup docs  https://review.openstack.org/56167222:40
openstackgerritMerged openstack-infra/zuul master: Update test instructions  https://review.openstack.org/56228922:40
clarkbya looks like its tested as both a list and a string, I think it may just be the yaml structure off earlier22:41
SpamapSohhhhhhh22:41
SpamapSok22:41
SpamapSit's the name: thing22:41
SpamapSshould be deploy-to-prod:22:41
clarkbya22:41
SpamapShttp://paste.openstack.org/show/719672/22:45
SpamapSfixed it22:45
*** JasonCL has quit IRC22:48
clarkbfungi: https://review.openstack.org/#/c/554352/6 looks ready except for tobiash's comment, do you want to do a quick update on that?22:55
* fungi catches back up after a very fine blackened sheepshead fillet23:09
fungiclarkb: corvus had mentioned he was planning to add his ssh key there to fix that todo. my concern is keeping some reminder for ourselves that we should have a minimum of 3 contacts listed there23:10
clarkbah23:11
corvusclarkb: re timezone -- yeah, i think merging 562273 is the only thing we can do.  folks with servers in utc (like openstack) will be unchanged, but folks with localized timezones will have a gregorian event.23:11
clarkbcorvus: it will soon pass fortunately23:12
corvusclarkb: yeah, though that reminds me that expiration is still unimplemented :)23:12
clarkbI'm currently trying to understand zuu's symlinking of roles so I can approve this tobiash bugfix23:12
clarkbcorvus: tobiash in https://review.openstack.org/#/c/560554/2//COMMIT_MSG the error message seems to indicate to me that it did use the name ansible and that didn't get truncated to ''23:15
corvusclarkb: i believe there's supposed to be an "ansible" at the endo f the role_0 thing23:16
clarkbcorvus: oh, the arrow points in the wrong direction23:17
clarkbthat is not confusing at all python23:17
corvusclarkb: nope.  i feel it's par for the course with symlinks.23:17
clarkbI'm going to test that really quickly locally but ya that makes sense if that is what it is23:17
clarkbthat is totally what it is23:18
corvusclarkb: yeah, do test, but iirc, that's supposed to be creating a directory (role_0) with one item in it (ansible) which is a symlink to a repo.  then role_0 gets added to the roles path, so if you do "roles: ansible" it finds that in the path23:18
clarkbimo that arrow should point the other way since it is creating a link role_0 to target ansible23:19
clarkbnot target ansible to link role_023:19
corvusagreed23:20
corvusthat set me back like 2 hours when debugging this originally23:21
corvuserror messages 101: it's better to say nothing than say the opposite of the truth23:22
corvusclarkb: one thing about mappingproxy:  isinstance(types.MappingProxyType({}), dict) == False23:24
clarkbhrm23:26
clarkbI guess from a is it a duck standpoint that is true23:26
clarkb(it has a reduced api)23:26
clarkband can't quack23:26
corvusyeah.  but i only care if, um, it has feathers.23:26
fungipeking duck has no feathers23:27
corvusso, it makes things like traversal difficult (where you do "if list; elif dict; else ...")23:27
corvusfungi: but lots of fat, and so do pigs...23:27
fungimmm, duck/porkfat23:27
corvus(at least, it's *supposed* to have lots of fat)23:27
openstackgerritMerged openstack-infra/zuul master: Store build start/end time in UTC  https://review.openstack.org/56227323:27
clarkbhrm I kinda awnt duck now23:28
clarkbmaybe that is what I can grill this weekend23:28
clarkbor smoke23:28
fungismoked duck might turn out really tasty23:29
corvus++23:31
openstackgerritMerged openstack-infra/zuul master: Fix implicit role for repos named ansible  https://review.openstack.org/56055423:34
corvusokay, i think i've ended up getting all the deepcopies out of the model (except on the unparsedtenantconfig, but that's fine for now, and it'll go away when we fully reuse config objects.  i think)23:34
corvusi also found a solution to actually make mappingproxy deep-copyable, but i'd prefer not to use it23:34
openstackgerritJames E. Blair proposed openstack-infra/zuul master: WIP: Make config objects freezable  https://review.openstack.org/56281623:35
clarkbcorvus: tobiash I've approved https://review.openstack.org/#/c/559745/3 but also noted a potential cleanup there23:39
clarkbcorvus: tristanC for the mqtt connection stuff, do we really need to explicitly set the tls versions? isn't that something clients typically do based on what the server supports?23:41
clarkbfeel similar to the ipv6 + statsd thing. Is this a knob that we really need or can it be determined at runtime23:41
clarkb(and we'll haev to update zuul everytime tls makes a new version)23:42
corvusclarkb: yeah, i like the extra caution around the release since it's in an exception handler.  it's a bit more future proof.23:43
corvusclarkb: maybe a q for mtreinish?23:44
clarkb"By default (if the python version supports it) the highest TLS version is detected. If unavailable, TLS v1 is used." says the docs23:46
clarkbso I guess the issue will be if the server supports older TLS than the client23:47
corvusclarkb: sounds like it doesn't have a fallback process?23:49
corvusie, a negotiation23:49
clarkbhttps://docs.python.org/3/library/ssl.html#ssl.PROTOCOL_TLS is the negotiation process I think23:49
clarkbwhich it falls back on if tlsv1.2 fails23:50
clarkboh wait hrm23:50
clarkbI think it would be worth testing using ^23:50
clarkboh taht is new in python3.6?23:51
openstackgerritMerged openstack-infra/zuul master: Fix missing semaphore release on zk error  https://review.openstack.org/55974523:51
clarkbno it says that but its in 3.5 docs too23:52
clarkbI think we may want to use that and then exclude ssl23:52
clarkbhttps://github.com/eclipse/paho.mqtt.python/blob/master/src/paho/mqtt/client.py#L684-L69023:53
corvusclarkb: it's also in the 2.7 docs23:54
clarkbmy reading of that is we maybe don't need to worry about tls versions since we require python3.5 anyways which should be new enough to support that23:54
corvushttps://docs.python.org/2.7/library/ssl.html#ssl.PROTOCOL_TLS23:54
corvussomething's really weird with that23:54
corvusclarkb: i agree this is worth holding the change for23:55
clarkbthe difference between python3.6 and python3.5 is 3.6 won't use sslv3 or v2 at all23:56
clarkbbut 3.5 will if tls isn't supported23:56
clarkbsorry 3.5 will only use sslv3 if not tls, it will not use sslv223:56
clarkbI think what we want to do is rely on the PROTOCOL_TLS constant and then maybe do a pull request to paho mqtt to disable SSLv3 at https://github.com/eclipse/paho.mqtt.python/blob/master/src/paho/mqtt/client.py#L690 using https://docs.python.org/3.5/library/ssl.html#protocol-versions or even https://docs.python.org/3.5/library/ssl.html#ssl.create_default_context23:58
clarkb(I'm not super concerned about us accidentally running an mqtt without tls support though23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!