From: Jude N Date: Fri, 21 Sep 2018 12:21:45 +0000 (-0400) Subject: Checkpoint X-Git-Url: https://pwan.org/git/?p=pwan.org.git;a=commitdiff_plain;h=6b4cb8e61e634b989a887c60fdd26d7140a8224f Checkpoint --- diff --git a/content/hints/elixir-on-centos7.rst b/content/hints/elixir-on-centos7.rst new file mode 100644 index 0000000..5529622 --- /dev/null +++ b/content/hints/elixir-on-centos7.rst @@ -0,0 +1,53 @@ +Elixir on Centos 7 +################## + +:date: 2017-10-08 +:tags: hints, elixir, centos +:category: hints +:author: Jude N + +I'm at the point with a Elixir/Phoneix side project that I'm thinking about deployment. + +The first big stumbling block was that development envioronment (Ubuntu 16.04) wasn't the same as my deployment environment (Centos 7). +For Ubuntu, I could pull the latest Elixir packages from `Erlang Solutions`_, but they don't host Centos Elixir packages, +and the version of Elixir on EPEL is `over 4 years old`_ - old enough that 'mix deps' on my project was erroring out. + +I found the posts below about installing Elixir on Centos 7 but, they involve cloning the Elixir repo and building it from source, +but I don't want development tools like git and gcc on my production machines. + +- https://gist.github.com/binarytemple-clients/52bec05fc9bdba80d737 +- http://www.jeramysingleton.com/install-erlang-and-elixir-on-centos-7-minimal/ +- https://www.unixmen.com/install-erlang-elixir-centos-7/ + +I also found https://www.vultr.com/docs/how-to-install-the-phoenix-framework-on-centos-7 but it involves downloading a +precompiled `mystery meat`_ bundle off Github. However https://github.com/elixir-lang/elixir doesn't mention that these precompiled +bundles are available or how the bundles were built. The precompiled bundle is mentioned on `the elixir-lang.org install page`_, but still, that's too mysterious for me. + +Maybe one of those links will work for you, but `what I want`_ is a way to build a more recent version of the Elixir rpm than what was available in EPEL. That way I can recompile and package Elixir on a dev machine, and then only copy the rpm up to my production machine. + +It looks like Elixir is stuck in EPEL waiting for the Erlang 18 release to get promoted in EPEL, so maybe I can take the existing Elixir packaging source and build it against the latest Erlang packages from Erlang Solutions... + +I found the packaging source at https://src.fedoraproject.org/rpms/elixir and after poking around a bit I came up with Vagrantfile below. It seems to be working OK so far. +: + +.. raw:: html + + + +Lessons Learned +--------------- +- `spectool`_ is my new favorite utility. +- Make sure your Elixir development environment is as close as possible to your deployment environment. + +Next Steps +---------- +- Convert the Vagrantfile into a Dockerfile or start using vagrant-lxc + +.. _Erlang Solutions: https://www.erlang-solutions.com/resources/download.html +.. _over 4 years old: https://koji.fedoraproject.org/koji/buildinfo?buildID=552725 +.. _the elixir-lang.org install page: https://elixir-lang.org/install.html#precompiled-package +.. _mystery meat: http://www.torontosun.com/2017/07/19/mystery-meat-crashes-onto-florida-familys-roof +.. _spectool: http://pkgbuild.sourceforge.net/spectool.html +.. _what I want: https://www.youtube.com/watch?v=gJLIiF15wjQ + + diff --git a/content/hints/lua-macros-in-rpm-specfiles.rst b/content/hints/lua-macros-in-rpm-specfiles.rst new file mode 100644 index 0000000..f4e4379 --- /dev/null +++ b/content/hints/lua-macros-in-rpm-specfiles.rst @@ -0,0 +1,17 @@ +Using Lua macros in an RPM specfile +################################### + +:date: 2015-07-12 +:tags: hint,lua,rpmbuild,rpm,specilfe +:fcategory: hints +:author: Jude N + +I've found using Lua macros in rpm spec files to be pretty useful. I haven't found many examples of their use online, so here's how I ended up using them. + +I had a situation where I needed to make a subpackage from all the files in a number of different subdirectories. The structure of the files in the subdirectories was fixed, +but the number of subdirectories could change over time, and I didn't want to have to update the spec file each time a new subdirectory was added or removed. + + + + +.. _lua macros in rpm spec files: http://www.rpm.org/wiki/PackagerDocs/RpmLua diff --git a/content/hints/pm-utils-ubuntu-vagrant-fun.rst b/content/hints/pm-utils-ubuntu-vagrant-fun.rst new file mode 100644 index 0000000..0c05d43 --- /dev/null +++ b/content/hints/pm-utils-ubuntu-vagrant-fun.rst @@ -0,0 +1,82 @@ +Using pm-utils to save/restore VMs on workstation suspend/restore +################################################################# + +:date: 2017-09-16 +:tags: hints,ubuntu,vagrant,pm +:category: hints +:author: Jude N + +I use Ubuntu (16.04 for now) and Vagrant (1.9.0 for now) on a bunch of my projects, and I've been running into somethinng like this +`power management bug` for a while now, where after restoring from suspension, my vagrant sessions would be dead and I'd have to +'vagrant halt' and 'vagrant up' before another 'vagrant ssh' would succeed. + +To work around this, I came up with some /etc/pm/sleep.d scripts which would save any running vagrant boxes when suspending the +workstation and then resume the VMs when resuming the workstation. + +Now if I'm in a 'vagrant ssh' session and Ubuntu suspends/resumes, instead of coming back to a frozen session, I'll see I've been +disconnected from the ssh session, and I can do another 'vagrant ssh' without having to halt/re-up the VM. That's better than nothing, +but the next step here is to start using something like screen or tmux in my vagrant sessions so I can restore right back to where I left +off. + +So why bother with two scripts when you could have 1 script with a single case statement ? I wanted saving the running vagrant boxes to +happen when all the usual services and userspace infrastructure was still running, so I wanted that script in the 00 -49 range from as +per the 'Sleep Hook Ordering Convention' portion of 'man 8 pm-action`. However I don't restoration to happen until all the services +restarted, so I pushed to the end of the service handling hook range. I may want to revisit this, and rename it to 75_vagrant. + +Note in the resume script, the command is pushed into the background since I didn't want want to wait for the VMs to be restored before +resuming Ubuntu. I'm usually checking email or the web for a bit before going back to my VMs so I'm OK if that's ready immediately. + +Here are some other lessons I learned from these scripts: + +- I didn't know `vagrant global-status`_ existed ! +- vagrant commands have to be run as the user started the VM. I had to use runuser to run the vagrant commands as my dev user. +- I learned the '-I %' approach of `getting xargs to insert it's argument in the middle of a command`_. + +The first script is /etc/pm/sleep.d/01_vagrant: + +.. code-block:: bash + + #!/bin/bash + + YOURNAME="your normal nonroot user name" + + case "$1" in + suspend) + timestamp=`date --rfc-3339=seconds` + echo "${timestamp}: $0 output" >> /var/log/pm-suspend-vagrant.log + (/sbin/runuser -u ${YOURNAME} /usr/bin/vagrant global-status | grep running | awk '{ print $1; }' | xargs -L1 -I % runuser -u ${YOURNAME} vagrant suspend % ) >> /var/log/pm-suspend-vagrant.log + ;; + *) + ;; + esac + + # Don't let errors above stop suspension + true + +The second script is /etc/pm/sleep.d/74_vagrant.sh + +.. code-block:: bash + + #!/bin/bash + + YOURNAME="your normal nonroot user name" + + case "$1" in + resume) + # Push the restoration into the background so it doesn't slow down + timestamp=`date --rfc-3339=seconds` + ((/sbin/runuser -u ${YOURNAME} /usr/bin/vagrant global-status | grep saved | awk '{ print $1; }' | xargs -L1 -I % runuser -u ${YOURNAME} vagrant resume % ) >> /var/log/pm-resume-vagrant.log) & + ;; + *) + ;; + esac + + # Don't let errors above stop restoration + true + +Sources: +- http://manpages.ubuntu.com/manpages/xenial/man8/pm-action.8.html + +.. _power management bug: https://www.virtualbox.org/ticket/15378 +.. _vagrant global-status: https://www.vagrantup.com/docs/cli/global-status.html +.. _getting xargs to insert it's argument in the middle of a command: https://stackoverflow.com/a/40355248/496127 diff --git a/content/lessons/Certmaster.rst b/content/lessons/Certmaster.rst new file mode 100644 index 0000000..0be6718 --- /dev/null +++ b/content/lessons/Certmaster.rst @@ -0,0 +1,36 @@ +Certmaster lessons learned +########################## + +:date: 2015-12-18 +:tags: lessons,certmaster +:category: lessons +:author: Jude N + +I'm started occassional series about lessons I've learned after finishing with a project. The kick-off article is about a `certmaster`_ fork I've been working on. +I've been using certmaster at $work for a few years now, but when we wanted to start using multiple certificate authorities, we had to spin up different instances of certmaster, with each instance +using on its own IP/port. It would be better if a single instance of certmaster could serve multiple CA's by adding a '--ca' flag. This is the functionality that `my fork of certmaster`_ provides, and +here are the lessons I learned while working on this: + +bats versus shunit2 +------------------- +certmaster doesn't include any tests, so I wanted to write some functional tests to verify my changes worked as expected. + +I started out working with `bats_`, but it fell down when I needed to push a command into the background - it just wouldn't do it. I tried the 'gotcha' suggestions from `this engine yard post`_ but to no avail. +I switched the tests to `shunit2`_ and had no trouble pushing commands into the background. + +Assigning here documents to variables +------------------------------------- + +.. code-block:: bash + + variable=$(cat < netmask + route add default gw + echo "nameserver 8.8.8.8" >> /etc/resolv.conf + + +Partitioning the disk +--------------------- +I spent a lot of time messing with various partitioning schemes until I stumbled across one that worked. I didn't need disk encryption, and I didn't want to bother trying `UEFI with ovirt`_, so here's what I ended up with. + +- A 20G disk split into /dev/sda1 and /dev/sda2 +- /dev/sda1 is a 400MB 'WIN VFAT32' partition (type 'b', not type '4' !!) +- /dev/sda2 is a LVM partition with the rest of the space +- For the LVM, /dev/vg/swap is an 8G swap partition and /dev/vg/root has the rest of the LVM parition + +In retrospect, I think a lot of my partitioning pain may have been caused by trying to have /dev/sda1 set as a BIOS Parition (type '4'), since I suspect the BIOS partition has to be under 32M. + +Also in retrospect, I see only 23M is actually used on the current /boot parition, so maybe 400MB was way too much and I should have gone with /dev/sda1 being 32M and type '4'. ¯\\_(ツ)_/¯ + +I think I also ran into problems using fsck on the boot partition instead of fsck.vfat. + +When the boot partition wasn't working, grub would fall into rescue mode and the various 'set prefix / set root / insmod' fixes like `this one`_ or `this other one`_ didn't work. + +What did work here was booting the system with the install CD again, mounting /mnt/boot manually and seeing that failed, or that /mnt/boot contained gibberish after mounting, and then unmounting /mnt/boot and using `testdisk`_ to fix the partition type. Testdisk really saved the day. + +Mounting the boot partition +--------------------------- +Before running nixos-install, I had to also mount the boot partition under /mnt/boot: + +.. code-block:: bash + + > mount /dev/vg/root /mnt + > mkdir -p /mnt/boot + > mount /dev/sda1 /mnt/boot + > nixos-install + + +Verify the /mnt/etc/nixos/hardware-configuration.nix device paths +----------------------------------------------------------------- +When I was messing with the disk partitioning, I rebuilt the /dev/sda1 partition a couple times. Apparently when you do that, you get new UUID for the device. + +This meant the "/boot" file system in /mnt/etc/nixos/hardware-configuration.nix was using a device path that was no longer valid. I updated the file to point to the current /boot device and reran 'nixos-install'. + +It looks like nixos-install isn't verifying the device paths are valid, since nixos-install ran OK with the invalid device paths. + + +Configuring a static IP in /mnt/etc/nixos/configuration.nix +----------------------------------------------------------- +Here's what I ended up adding to the configuration.nix file to set up static IP: + +.. code-block:: bash + + networking = { + hostName = ''; + usePredictableInterfacenames = false; + interfaces.eth0.ip4 = [{ + address= ""; + prefixLength = ; + }]; + defaultGateway = "" + nameservers = [ "8.8.8.8" ]; + }; + +I also adding this boot setting: + +.. code-block:: bash + + boot.load.grub.device = "/dev/sda"; + +Sources +------- +- https://chris-martin.org/2015/installing-nixos (useful, but I didn't want UEFI or disk encryption - I got the 'have to mount /mnt/boot' step here) +- https://gist.github.com/martijnvermaat/76f2e24d0239470dd71050358b4d5134 +- https://polycrystal.org/posts/2014-10-27-installing-nix.html + +.. _NixOS: https://nixos.org/ +.. _UEFI with ovirt: https://bugzilla.redhat.com/show_bug.cgi?id=1327846 +.. _this one: https://askubuntu.com/questions/119597/grub-rescue-error-unknown-filesystem +.. _this other one: https://www.linux.com/learn/how-rescue-non-booting-grub-2-Linux +.. _testdisk: http://www.cgsecurity.org/wiki/TestDisk diff --git a/content/lessons/PylintPlugin.rst b/content/lessons/PylintPlugin.rst new file mode 100755 index 0000000..5681fe1 --- /dev/null +++ b/content/lessons/PylintPlugin.rst @@ -0,0 +1,75 @@ +Lessons from Wriing a Pylint Plugin +################################### + +:date: 2017-09-24 +:tags: lessons,python,pylint,peer reviews +:category: lessons +:author: Jude N + +At work there's a python coding convention that I tend to overlook a lot. +So when I post merge requests, there's a pretty good chance someone's going to +call me out on this, which leads to a followup commit and another round of +peer review. This can lead to an extra delay of a few hours until I notice +the comments, switch context back to that merge request, making the changes, +update the merge request and wait for another round of reviews. If I could +find a way to check my code for this convention before posting the merge +requsts, I could get my code merged in a few hours faster.... + +The Convention +-------------- + +The coding convention I cannot internalize is as follows: In python, +the format method for strings will call the __format__ method on its arguments +for you, so any code that looks like: + +.. code-block:: python + + "interpolate these: {} {}".format(str(a), str(b)) + +Need only look like: + +.. code-block:: python + + "interpolate me: {} {}".format(a, b) + +The Pylint Plugin +----------------- +So googling around led my to `this Ned Batchelder post`_ from a few years back. +That post also led to a couple pylint plugins `here`_. Looking at `pylint's own format checker`_ +reminded me that I should also be handling keyword arguments. + +From the post and sample code, it looked like I needed to define a checker class with a visit_callfunc method +that would check when the 'format' method was used, and then check all the arguments to the format call and +throw an error if any of them where a function call to str(). + +Here's what I eventually ended up. + +.. raw:: html + + + + +To come up with this I used an embarassing amount of `exploratory programming`_ to figure out astroid. I wrote an initial +:code:`visit_callfunc()` method based on the sample code that didn't do much more than dump out all the data about the node argument via +:code:`dir(node)` and :code:`node.__dict__`. +Then I would call pylint with the plugin against some sample source with the error I was trying to plugin to report. + +I run the plugin against the existing code and found one lingering case where the reviewers had allowed one of my unneccessary str() +call into the codebase. It's been removed now. + + +Lessons Learned +=============== + +- pylint plugins are pretty powerful and I wouldn't shy away from writing another one. I'm on the lookout for other excuses to write another one. +- https://greentreesnakes.readthedocs.io is a useful 'missing manual' for the python AST. +- format() can take both positional and keyword arguments. My original pass at the plugin only supported positional arguments. +- The `bandit`_ project exists and looks useful. I stumbled acros it while looking for other pylint plugins. + + +.. _this Ned Batchelder post: https://nedbatchelder.com/blog/201505/writing_pylint_plugins.html +.. _here: https://github.com/edx/edx-lint/tree/master/edx_lint/pylint +.. _pylint's own format checker: https://github.com/PyCQA/pylint/blob/master/pylint/checkers/strings.py#L227 +.. _exploratory programming: http://wiki.c2.com/?ExploratoryProgramming +.. _bandit: https://wiki.openstack.org/wiki/Security/Projects/Bandit + diff --git a/content/pages/resume.rst b/content/pages/resume.rst index 6b24a8c..df10abe 100644 --- a/content/pages/resume.rst +++ b/content/pages/resume.rst @@ -1,32 +1,32 @@ Jude Nagurney ############# -:date: 2015-02-12 -:tags: resume -:category: resume -:author: Jude N -:status: hidden +.. :date: 2018-09-20 +.. :tags: resume +.. :category: resume +.. :author: Jude N +.. :status: hidden | 1136 Forest Edge Drive | Reston, Virginia, 20190 -| Phone: (703) 831-7492 +| Phone: (703) 829-0744 | jude.nagurney@gmail.com ======= Summary ======= -| I'm a results-oriented software engineer with a strong focus on agile and devops-related processes. +| I'm a results-oriented software engineer with a strong focus on agile and devops processes. ================= Technical Skills: ================= -- **Languages** : Python, C/C++, Ruby, Perl, Java, SQL, Bash, Expect, Tcl/Tk, UML OCL, COBOL -- **Tools** : Puppet, Cobbler, Jenkins, emacs, vi, Jira, Bugzilla, git, Mercurial, Subversion, Jira, Crucible, Reviewboard, xUnit, SELinux -- **Frameworks** : Django, Pylons, Rails, TurboGears, SqlAlchemy -- **Operating Systems** : Linux (Ubuntu, RedHat, CentOS), Microsoft Windows, vxWorks, Solaris -- **Databases** : PostgresSQL, MySQL, Oracle, DB2 +- **Languages** : Python, C/C++, Ruby, Perl, Java, SQL, Bash, lua, Expect, Tcl/Tk, UML OCL, COBOL +- **Tools** : Puppet, Salt, Cobbler, Jenkins, emacs, vi, Jira, git, Gitlab, Docker, Mercurial, Subversion, Jira, SELinux +- **Frameworks** : Django, Angular, Pylons, Rails, SqlAlchemy +- **Operating Systems** : Linux (Ubuntu, Debian, RedHat, CentOS, Raspbian), Microsoft Windows, vxWorks, Solaris +- **Databases** : PostgresSQL, MySQL, Oracle - **Standards Expertise** : SONET, SDH, TL1, LMP ---- @@ -35,15 +35,53 @@ Technical Skills: Work Experience =============== +------------------ +Layer 2 Technology +------------------ +| **Reston Virginia** +| **October 2016 - Present** + +----------------- +Software Engineer +----------------- + +Developed and maintained Python-based software projects + +- Developed SMS-based solutions using services such as Twilio, Plivo, Nexmo, and Vitelity. + +- Wrote a Errbot plugin for reporting open merge requests that were waiting for peer reviews. + +- Worked on porting projects to Raspbian to run on a Raspberry Pi 3 Model B. + This included rebuilding packages for the arm7 architecture. + +- Developed and maintained Salt states and Puppet manifests for various projects. + +- Developed and maintained Jenkins continuous integration jobs for various projects. + Also proactively tracked down the root causes of build failures when the jobs failed. + --------------------- Applied Security Inc. --------------------- | **Reston Virginia** -| **March 2010 – Present** +| **March 2010 – October 2016** + +..................................... +Software Engineer (Development Group) +..................................... +| **April 2016 - October 2016** + +Wrote Python code for new projects and extended existing Python code bases + +- Extended a project to dynamically allocate AWS hosts based on system usage. + +- Wrote Python code for sending and receiving SMS messages through Plivo and Twilio + +- Developed and maintained Puppet manifests for development projects .................................. Software Engineer (Security Group) .................................. +| **March 2014 - April 2016** Extended devops practices to cover security reviews @@ -55,6 +93,7 @@ Extended devops practices to cover security reviews ..................................... Software Engineer (Engineering Group) ..................................... +| **March 2012 - March 2014** Introduced 'infrastructure-as-code' to the ASI Engineering group. @@ -69,6 +108,7 @@ Introduced 'infrastructure-as-code' to the ASI Engineering group. ........................................ Software Engineer (Web Technology Group) ........................................ +| **March 2010 - March 2012** Managed, developed and maintained the Web Technology infrastructure environment. @@ -329,7 +369,6 @@ Education: ---------------------------------------------------------------------- George Mason University, Information Technology and Engineering School ---------------------------------------------------------------------- -Masters of Science / Systems Engineering / Expected Graduation May 2014 Masters of Science / Computer Science / May 2006 ------------------------------------------ @@ -342,6 +381,9 @@ Bachelor of Science / Computer Science / May 1990 Open Source Projects: ===================== +Certmaster (http://github.com/jude/certmaster) (2015-Present) +- Forked the Fedora Certmaster project adding support for multiple CAs and hash functions other than sha1 + Haskell Augeas FFI Bindings (http://trac.haskell.org/augeas/ (2009-Present) - Provided foreign function interface bindings so Haskell users could easily use the Augeas library diff --git a/content/recipes/blizzard-chow.rst b/content/recipes/blizzard-chow.rst new file mode 100644 index 0000000..d829d93 --- /dev/null +++ b/content/recipes/blizzard-chow.rst @@ -0,0 +1,49 @@ +Blizzard Chow +############# + +:date: 2016-01-25 +:tags: recipes +:category: recipes +:author: Jude N + +I came up with the receipe below while `making the most of my blizzard food stash`_. + +----------- +Ingrediants +----------- + +- 1 lb ground turkey + +- ~4 medium celery stalks, diced + +- ~4 medium carrots, diced + +- 1 onion, chopped + +- ~6 garlic cloves, chopped + +- 1tsp `Chinese five spice`_ + +- 1tsp basil + +- 1tsp oregano + +----- +Steps +----- + +1) Brown ground turkey and leave on medium heat. +2) Heat the garlic and onion until the onion is clear. +3) Add garlic, onion, and spices to the turkey. +4) Heat the rest of the veggies in a separate pan. +5) When the veggies are tender, add half of them to the ground turkey. +6) Put the otiher half in a blender and blend them into a sauce. +7) Add the veggie sauce to the ground turkey. +8) Adjust sauciness by adding a cup or 2 or water to the ground turkey. +9) Continue heating until you're convinced the turkey is finished cooking. +10) Salt & pepper to taste. + +I only blended half the veggies because the pan I was using was too small to cook all the veggies at once, but it turned out really nice, especially with some chipotle hot sauce. + +.. _making the most of my blizzard food stash: https://www.washingtonpost.com/lifestyle/food/how-to-make-the-most-of-your-blizzard-food-stash--even-when-you-lose-power/2016/01/22/d08d2572-c10c-11e5-bcda-62a36b394160_story.html +.. _Chinese five spice: https://en.wikipedia.org/wiki/Five-spice_powder