Monday, 2014-01-27

openstackgerritRuslan Kamaldinov proposed a change to openstack-infra/storyboard: Added documentation for REST API layer.  https://review.openstack.org/6921200:02
*** drdee has joined #storyboard02:16
*** drdee has quit IRC03:21
*** drdee has joined #storyboard03:37
*** drdee has quit IRC03:43
*** david-lyle has joined #storyboard03:51
*** drdee has joined #storyboard03:55
*** drdee has quit IRC04:22
*** drdee has joined #storyboard04:28
*** drdee has quit IRC05:47
*** SergeyLukjanov_ is now known as SergeyLukjanov06:04
*** SergeyLukjanov is now known as SergeyLukjanov_06:31
*** miqui has quit IRC06:33
*** miqui has joined #storyboard06:33
*** SergeyLukjanov_ is now known as SergeyLukjanov06:57
*** SergeyLukjanov is now known as SergeyLukjanov_07:20
*** SergeyLukjanov_ is now known as SergeyLukjanov07:48
*** SergeyLukjanov is now known as SergeyLukjanov_08:03
*** SergeyLukjanov_ is now known as SergeyLukjanov08:03
*** SergeyLukjanov is now known as SergeyLukjanov_08:17
*** che-arne has quit IRC08:40
openstackgerritNikita Konovalov proposed a change to openstack-infra/storyboard: Load projects from yaml file  https://review.openstack.org/6628009:18
*** _ruhe is now known as ruhe09:35
ttxilyashakhat, SergeyLukjanov_, NikitaKonovalov_, ruhe : progress on the visa ?10:46
*** NikitaKonovalov_ is now known as NikitaKonovalov10:47
ruhettx: SergeyLukjanov_, NikitaKonovalov_, ruhe still don't have visas. our chances aren't good10:47
ttxdamn10:47
ruheyeah, turns out belgium embassy has a bad reputation here for not allowing people to enter without any reason10:48
ttxruhe: if there is anything I can do to help, let me know10:48
ruhettx: thank you. our HR do the best they can, but they can't even request current status of our visas. we only know they're still on review.10:49
ruhethere is a claim on the website that it take no more than 10 days to process visa application. it's already past 20 working days since we applied10:50
ttxruhe: was looking if france would be more accommodating, but apparently you need to apply in the country you'll stay in (which is Belgium) so that wouldn't help10:51
ruhettx: well, in case if we don't make it, you guys please scrupulously document everything you agree on :)11:00
*** ruhe is now known as _ruhe11:12
*** SergeyLukjanov_ is now known as SergeyLukjanov11:17
*** SergeyLukjanov is now known as SergeyLukjanov_11:33
*** _ruhe is now known as ruhe11:39
openstackgerritNikita Konovalov proposed a change to openstack-infra/storyboard: Load projects from yaml file  https://review.openstack.org/6628012:01
*** ruhe is now known as _ruhe12:52
*** _ruhe is now known as ruhe12:54
*** david-lyle has quit IRC13:05
*** SergeyLukjanov_ is now known as SergeyLukjanov13:31
ttxlooks unstuck but the graph really looks funny13:49
*** miqui has joined #storyboard14:09
*** drdee has joined #storyboard14:18
*** drdee has quit IRC14:39
openstackgerritRuslan Kamaldinov proposed a change to openstack-infra/storyboard: Update documentation  https://review.openstack.org/6921114:54
mordredmorning all15:06
* mordred is sad about visas :(15:06
mordredperhaps next time we should pick a country that's easy for both russians and americans ... I think finland is friendly to both? and latvia?15:07
mordredruhe: I'm going to spin up a database and a cloud server to try your puppet out on15:07
mordredruhe: can you fix one thing real quick? (or I can) - the hostname in site.pp should not be capitalized15:08
*** ruhe is now known as _ruhe15:12
*** cody-somerville has joined #storyboard15:17
openstackgerritNikita Konovalov proposed a change to openstack-infra/storyboard: Database fixture added  https://review.openstack.org/6938415:21
mordred_ruhe: nevermind - I needed to make a couple of other small changes too15:28
*** _ruhe is now known as ruhe15:33
mordredruhe: I'm rebasing your change on top of the projects.yaml rework15:35
ruhemordred: can you point me to patch? :) which one you referring to?15:36
mordredNikitaKonovalov: your load-from-yaml patch is WIP - anything I can help poke at there?15:37
mordredruhe: the puppet patch15:37
ruhemordred: ah, this is my favourite patch15:37
mordredruhe: I'm about to test it out on a new server :)15:38
mordredruhe: I also changed it to consume an external db15:38
ruhemordred: awesome!15:38
*** SergeyLukjanov is now known as SergeyLukjanov_15:38
*** david-lyle has joined #storyboard15:42
*** drdee has joined #storyboard15:45
miqui..hello...15:48
miquiruhe: am curious did you build a new puppet type to read an external db? say like mysql?15:49
ruhemiqui: mordred already did that15:49
ruhemordred: is it going to be Trove managed MySQL database?15:50
mordredyes15:58
mordredso it actually means it's easier - we don't have to manage the db in puppet at all15:58
*** SergeyLukjanov_ is now known as SergeyLukjanov15:58
*** NikitaKonovalov is now known as NikitaKonovalov_16:00
ruhemordred: one more question. you added $projects_file, but there is no action to execute "storyboard-db-manage load-projects". is it intentional?16:01
mordredwell, I wanted to go ahead and add the file - but then the load-projects patch hasnt' landed yet16:03
ruhemordred: i still -1 that patch because it lacks test coverage. i don't want something untested to be used every day in production16:04
ruheNikita already did a good job, soon he will upload testing infrastructure and all these stuff will be tested16:05
mordredcool16:06
mordredruhe: well, I'm testing the puppet right now on the nwe server I created16:06
mordredruhe, ttx, NikitaKonovalov_: storyboard.openstack.org is now runing - it doesn't seem to do anything16:18
ruhemordred: did you fix webclient tarball url?16:20
ruhein init.pp line 2516:20
ruheat this moment there's only http://tarballs.openstack.org/storyboard-webclient/storyboard-webclient-master.tar.gz available16:21
ruhemordred: also http://storyboard.openstack.org/api/v1/projects returns 500 error. probably there is something wrong with storyboard-api16:22
ruhemordred: assuming index now display a nice page, i guess you fixed fix webclient tarball url16:23
ruheanyway, i'm happy storyboard.openstack.org is now online :)16:24
ruhei'll step away, for a couple of hours16:25
*** ruhe is now known as _ruhe16:25
openstackgerritMonty Taylor proposed a change to openstack-infra/storyboard: Don't try to install file that doesn't exist  https://review.openstack.org/6940216:28
*** SergeyLukjanov is now known as SergeyLukjanov_16:30
*** SergeyLukjanov_ is now known as SergeyLukjanov16:54
*** SergeyLukjanov is now known as SergeyLukjanov_17:03
openstackgerritA change was merged to openstack-infra/storyboard: Don't try to install file that doesn't exist  https://review.openstack.org/6940217:03
*** SergeyLukjanov_ is now known as SergeyLukjanov17:26
*** SergeyLukjanov is now known as SergeyLukjanov_17:41
openstackgerritMonty Taylor proposed a change to openstack-infra/storyboard: Added documentation for REST API layer.  https://review.openstack.org/6921217:42
*** NikitaKonovalov_ is now known as NikitaKonovalov18:02
*** NikitaKonovalov is now known as NikitaKonovalov_18:05
*** david-lyle has quit IRC18:06
*** NikitaKonovalov_ is now known as NikitaKonovalov18:09
*** krotscheck has joined #storyboard18:14
*** david-lyle has joined #storyboard18:21
NikitaKonovalovhi everyone18:25
* krotscheck is aghast. People have actually done code reviews. MADNESS!18:25
NikitaKonovalovthe log file told me, we've got a serrver running, correct?18:26
krotscheckNikitaKonovalov: No idea, I don't have a shell that stays connected to IRC overnight18:27
*** NikitaKonovalov is now known as NikitaKonovalov_18:27
*** _ruhe is now known as ruhe18:32
*** NikitaKonovalov_ is now known as NikitaKonovalov18:33
ruhekrotscheck: znc is a pretty good and easy to use irc bouncer18:34
ruheNikitaKonovalov: krotscheck: quote -   <mordred> ruhe, ttx, NikitaKonovalov_: storyboard.openstack.org is now runing - it doesn't seem to do anything18:35
NikitaKonovalovI'm using it right now, but it takes some time for messages to arrive18:36
ruhemordred: is there any way i could get access to server logs from storyboard.openstack.org?18:37
krotscheckkk18:38
krotscheckruhe: Neither http nor https are resolving for me.18:38
openstackgerritNikita Konovalov proposed a change to openstack-infra/storyboard: Database fixture added  https://review.openstack.org/6938418:40
krotscheckNetwork unreachable on storyboard.o.o - both from inside and outside of HP's network. It looks like it's trying to resolve IPv6 though18:40
NikitaKonovalovI cant get any response either18:41
krotscheckWell, I can ping the box. Ergo, we need server access to see what's going on. mordred's on a plane, his response time will be iffy18:42
ruhekrotscheck: NikitaKonovalov: it was browser cache. but i've seen it working :) at least there were html pages18:46
krotscheckruhe: Given that your patch still hasn't landed, it's probably safe to say that all the plumbing isn't ther eyet.18:47
krotscheck...or has it18:48
krotscheckNope.18:48
NikitaKonovalovok, then it's time to focus on load_projects patch, so that there is some data to display18:50
ruhekrotscheck: on the other hand, https://review.openstack.org/#/c/65017/ has +2 from mordred18:51
ruhewhich is huge success for me :)18:51
krotscheckruhe: Hellz yeah18:51
ruheeven though he added couple of changes to that patch18:51
krotscheckruhe: I'm currently updating the various artifact upload patches to use "latest"18:52
*** NikitaKonovalov is now known as NikitaKonovalov_18:53
*** SergeyLukjanov_ is now known as SergeyLukjanov18:53
*** NikitaKonovalov_ is now known as NikitaKonovalov18:57
*** NikitaKonovalov is now known as NikitaKonovalov_19:12
*** ruhe is now known as _ruhe19:15
openstackgerritMichael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Added no_api env  https://review.openstack.org/6861019:20
krotscheckNikitaKonovalov_- I just added some README text and renamed the node venv to venv, since that's hardcoded all over the place in infra.19:21
mordredmorning19:47
mordredkrotscheck: re: tox - we normally have an env that when run runs tests, and one called venv that doesn't do anything other than pass through commands to the venv - which we normally call venv19:52
krotscheckmordred: Interesting. Is that documented anywhere?19:52
mordredkrotscheck: https://wiki.openstack.org/wiki/ProjectTestingInterface19:54
mordredkrotscheck: now - that's python - so some things want to be different of course19:54
krotscheckUgh19:55
krotscheckmordred: Righto then19:56
krotscheckI can revert those. I'm currently working on the puppet module though.19:56
mordredkrotscheck: the current patch on the top of the stack seems to install properly - whatcha doing?19:58
krotscheckjeblair left some comments19:58
mordredcool19:58
* mordred goes to look19:58
mordredkrotscheck: so - the current issue I'm seeing is that when I try to go there , I get 503s, are you seeing that too?19:59
krotscheckmordred: You mean storyboard.o.o?20:00
krotscheckI'm not getting anything20:00
mordredoh. well that's different from me at least :)20:00
mordredI get Error 503 Service Unavailable from 166.78.175.2720:00
mordredBUT - if I curl the api url from a machien that is not my laptop , I get an empty list20:01
krotscheckThat COULD be correct.20:01
krotscheckI don't get an HTTP response from 166.78.175.27 at all20:01
krotschecktraceroute and ping work though, so that's a thing.20:02
krotscheckmordred: Would it make more sense to "curl $tarball | tar xvz" into /var/lib/storyboard/dist and then "rsync" from there to /var/lib/storyboard/www?20:03
mordredkrotscheck: yes. probably so20:04
krotscheckI don't know how rsync works with that race condition you mentioned.20:05
*** openstackgerrit has quit IRC20:06
*** openstackgerrit has joined #storyboard20:06
*** _ruhe is now known as ruhe20:06
mordredthe only way to be sure we have no race condition would be to untar into a new dir and then have /var/lib/storyboard/www be a synlink to that20:07
mordredso taht we can do ln -sf /var/lib/storyboard/dist/xxxxx /var/lib/storyboard/www20:07
mordredas an atomic action20:07
mordredHOWEVER - expressing that in puppet might be a bit crap20:07
mordredso we could just do rsync --delete from /var/lib/storyboard/dist to /var/lib/storyboard/www and be ok with the fact that there is a split second of time where we might have half of the html/js copied over and half of the old version20:08
cody-somervilleShouldn't we have the js files and stuff with a versioned url?20:10
mordredthat would very quickly become rather unworkable, no?20:10
mordredwe're talking about deploying every commit20:10
krotscheckcody-somerville: There's actually  a packaging step in grunt that appends part of the md5sum to the filename.20:11
mordredoh. nevermind. /me shuts up20:11
krotscheckcody-somerville: That's not enabled yet though, but it's a common strategy for cachebusting.20:11
* cody-somerville nods.20:11
ruhemordred: doesn't file { $unpack_target: purge   => true, } solve all the possible race errors?20:11
krotscheckSo rsync --delete is probably more dangerous.20:11
mordredruhe: does it? I thought puppet would just copy and then delete files - if it's smarter than that, then neat20:12
ruhemordred: it should just clear all the contents of the dir on each run because all the files in this directory aren't managed by puppet20:13
ruhei'm not 100% sure about this, but i can test it tomorrow (during night in US)20:14
krotscheckHey, shouldn't there be a service { 'httpd': ensure => running} somewhere?20:14
mordredI think that's handled by the apache class?20:14
ruhekrotscheck: mordred: yes, it should be handled by apache class20:15
krotscheckruhe: mordred: Got it20:15
ruhemordred: are you going to resolve comments from jeblair? or do you want me to resolve them?20:15
mordredruhe: I think krotscheck is20:16
krotscheckWorking on it.20:16
ruhekrotscheck: cool, thank you20:16
ruhecollaborative work produces best results20:16
*** drdee has quit IRC20:19
ruhekrotscheck: pleas note that file { $unpack_target: purge   => true, } might resolve race issues, but we need to test it20:20
ruhethere are folks (puppet gurus) in infra channel who can confirm it, but don't remember their nicks20:22
* mordred just asked AaronGr to pop in here20:25
*** AaronGr has joined #storyboard20:25
mordredAaronGr: we have questions aroudn the atomicity  of file { $unpack_target: purge   => true, }20:27
AaronGrmordred: what would you like to know?20:28
mordredruhe: meet AaronGr , he knows everything about puppet. also, my network is very laggy20:29
mordredkrotscheck: where can I find your ssh key? ruhe same question20:30
AaronGrruhe: monty lies, i know not nearly enough, but i'm working on it :)20:30
ruheAaronGr: hi!20:31
ruheAaronGr: so i have comments from jeblair here https://review.openstack.org/#/c/65017/14/modules/storyboard/manifests/init.pp20:31
ruhecomments about tmpdir races20:32
AaronGrruhe: ok, i can see those .. what's the question?20:33
mordredAaronGr: we want to lay out new html/js/css dirs from a tarball - and we'd like to do it atomicly20:34
mordredso that we dont' server half-old/half-new applications20:34
krotscheckmordred: Sec. Trying to resolve a bloody rebase conflict...20:38
mordredkrotscheck: enjoy that20:39
AaronGrruhe: sorry, i've gotta ask - what race condition is jeblair worried will happen?20:42
*** SergeyLukjanov is now known as SergeyLukjanov_20:44
AaronGrruhe: i can see the todo mentioning a race condition, what will definitely cause a breakage while it runs, but that's not what's being commented on.20:46
*** ruhe is now known as _ruhe20:46
krotscheckmordred: You removed review.projects.yaml?20:47
krotscheckmordred: n/m20:48
mordredAaronGr: two different issues - I believe jeblair is concerned about tempdir stuff - I'm worried about atomic updates20:50
AaronGrmordred: i would personally not use /tmp for these operations, since worst case it's a garbage can.  the rm/move is a bit ugly, that's definitely not atomic -- the operations in /tmp though look to be serial20:54
krotscheckAaronGr, mordred, _ruhe: I've just updated the patch.20:55
krotscheckAll changes are between line 119 and 153 (with a bit of a change at the top where I added curl as a package dependency)20:56
*** _ruhe is now known as ruhe20:56
krotscheckI took ruhe's suggestion to just use file{} rather than rsync20:57
mordredkrotscheck: yeah. worth a try!20:59
AaronGrkrotscheck: pre-lunch so my thought process isn't totally clear, but this looks like get-webclient will run every time -- is that intentional?20:59
mordredkrotscheck: so - one quick thing - how does that know there is a new webclietn tarball to pull?21:00
mordredAaronGr: jinx21:00
ruheyeah, it'll update webclient on each puppet run, which every 15 minutes if i'm not mistaken21:01
krotscheckOh, crap. Forgot that this'd be an agent run.21:01
AaronGrlooks like it's just curl -> unpack to folder. could you not do wget/curl to disk, then subscribe the unpack to that file?21:01
* krotscheck is still new to puppet21:01
AaronGrkind of a mix of the previous patch and this one21:01
ruhethis is the pain you get when you install from tarball21:03
AaronGrkrotscheck: it does deal with i/o blockage during an rm -- but it's still not atomic, since you are waiting on tar/gz21:03
AaronGrwait, i must need food. i see, unpacks to dist/, then puppet copies files around. yeah, that's better, other than the re-download.  back in a bit, food now.21:06
krotscheckI really just need to figure out how to only download the tarball when the Cache header changes.21:07
AaronGrkrotscheck: something like http://blog.yjl.im/2012/03/downloading-only-when-modified-using.html ?21:10
krotscheckAaronGr: Found the exact same flag on a different page, trying it now.21:12
krotscheckAaronGr: well, now it doesn't download every time, but it still triggers the untar and the file.21:27
krotscheckcurl -z still exits 0 if it doesn't download anything :/21:28
krotscheckI recall there being an option where it'll return how much it pulled down, seeing if I can trigge ron that...21:28
ruhehonestly, i did a lot of web search before i decided to use this ugly method: download->compare->unpack21:29
*** drdee has joined #storyboard21:32
*** miqui has quit IRC21:48
*** ruhe is now known as _ruhe21:48
krotscheck_ruhe: I'm (mostly) reverting to your method.21:56
*** drdee has quit IRC22:24
*** drdee has joined #storyboard22:27
krotscheckjesusaurus, you happen to know what that untar flag is off the top of your head?22:28
jesusaurusi think its just --delete22:28
jesusaurusbut i would have to open the manpage to double check22:29
*** _ruhe is now known as ruhe22:30
krotscheckjesusaurus: Yeah, that only works when creating an archive.22:31
jesusaurusoh...22:31
jesusauruskrotscheck: maybe --remove-files does what you want?22:33
*** ruhe is now known as _ruhe22:38
krotscheckmordred: So should I be changing the two tox reviews back to node rather than venv?22:46
mordredkrotscheck: I think we can call the other one whatever makes sense to you22:48
mordredkrotscheck: I think if venv doesn't take an argument, people will get confused22:48
mordredso, longwinded- yeah, I think so22:48
krotscheckkk23:08
krotscheckmordred: Ok, done.23:19
krotscheckmordred: Though it seems like those changes are dependent on a whole bunch of things.23:20
krotscheckmordred: or not.23:20
*** _ruhe is now known as ruhe23:22
ruhekrotscheck: re https://review.openstack.org/#/c/69204/3/modules/openstack_project/files/jenkins_job_builder/config/storyboard.yaml23:23
ruhei'd like to keep storyboard-docs jobs in file named storyboard.yaml23:23
ruheis it ok with you?23:23
openstackgerritMichael Krotscheck proposed a change to openstack-infra/storyboard-webclient: Added no_api env  https://review.openstack.org/6861023:23
krotscheckruhe: Yes, though I'm a big fan of making _all_ the docs builds use scp.23:25
ruheso it seems i need to base my patch on https://review.openstack.org/#/c/67731/23:30
mordredkrotscheck: we all are23:38
mordredkrotscheck, ruhe: https://review.openstack.org/69502 https://review.openstack.org/6950323:38
mordredI thought I'd landed those ages ago, seems I may have never uploaded them23:39
mordredthey're pre-reqs for landing the config patches to split the projects.yaml into projects.yaml and projects.ini23:39
mordredeyeballs would be kindly appreciated23:39
*** david-lyle has quit IRC23:40
*** drdee_ has joined #storyboard23:43
*** drdee_ has quit IRC23:44
*** ruhe is now known as _ruhe23:44
*** drdee has quit IRC23:48

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!