Category Archives: Windows

Linux Mint 18 “Sarah” MATE Edition on Acer Aspire E5-575-33BM

Just completed the installation of Linux Mint 18 on a new Acer Aspire E5-575-33BM as a Christmas present for my wife. Her 2007 Intel Core Duo Macbook 2,1 running LM 17.2 was getting too cranky and the battery life was getting shorter so it was time for an update/upgrade. I’d considered getting her a Chromebook but the price delta was not that much between getting a 15.6″ Chromebook (which has printing issues) and this machine which included a 1TB disk. She can always use my Chromebook by simply logging in with her Gmail account, so there was little to lose by going with the more capable machine.

She has become very comfortable with using Linux over the past year plus on her Mac and it has met all her computing needs so it was a no-brainer to avoid the intrusive solution that is Win 10 and go Linux on this unit too. The following are the steps I took to make it work. Got some clues from here and here.

  • Update Acer Firmware. The update tool works only under Windows, so I did this before modifying the machine. Booted into Win 10, did only a basic system set up (including disabling ALL automatic updates before connecting to the web) and then ran the FW update downloaded off the Acer site. Latest FW (jumping two versions) installed fine.
  • Twiddled some FW settings (press F2 at power on) to allow for Legacy mode booting off a Linux Mint 18 MATE live USB image and tested basic functionality – pretty much everything worked fine. Cool. Shut down and remove back panel to do some HW work.
  • Swapped out the Samsung EVO 850 SSD from her old Macbook to the new Acer, removing the 1TB disk for safe keeping with the Windows 10 install. Just swapped the applicable disk caddies and put the SSD in. For giggles I twiddled the FW once more to use Legacy (BIOS) mode and was able to boot the existing LM 17.2 image just fine on the new machine, but the wireless did not work and it was a 32 bit install, so it was time to install a new version for the 64bit machine.
  • Updated FW settings once more to enable USB boot for the installation and disabled Secure Boot (yet UEFI was enabled).
  • Booted live USB for installation. As for some strange reason I did not follow my common practice in the old LM 17.2 installation to set up a separate home partition, I needed to migrate the contents of home to someplace safe – so I used GParted to create and resize partitions for both home and the UEFI boot files (and to create backup images). Used FSArchiver (Partimage doesn’t work on ext4 file systems, which my prior install was on) to create a backup disk image of the old 17.2 environment. Migrated the home directory files using GRsync to the new /home partition. Installed LM 18 from the live USB, which took a surprisingly short time with a wired Ethernet connection.
  • Bring up FW set up once more and enabled Secure Boot again. Within FW “trust”ed the UEFI partition files for Secure Boot. Reboot and come up in new LM 18 environment. Using Driver Manager, installed the intel_microcode firmware to support the i3 processor. Reboot as required by Driver Manager.
  • Run Mint Update Manager and install all 167 offered updates. Onboard wireless works, but is incredibly slow (1 MB/s), yet all available updates are installed. Tried a bunch of online solutions to update the Atheros Ath10k firmware and kernel, etc. which didn’t work – and ended up causing some problems when I tried backing them out – so I ended up repeating the installation of Mint once more from the FW/Secure Boot settings through to the reboot required by Driver Manager above.

I have on hand an Edimax USB WiFi dongle with a Realtek chipset, so tried that one out via Network Manager and it connected at 54 MB/s right away. So for the time being, we’re sticking with that one and pretty much everything now works – sound, volume control via function keys, wireless, video playback, Jacquie Lawson Christmas Web Advent Calendar card (HTML5 or flash-based?). Brightness control does not work via the function keys yet but does work with the software control.

Machine is very snappy under Linux (much faster than it seemed under Windows 10) and idles with just single digit percentages of CPU core use and a small fraction of the 4MB DIMM capacity being used. Screen is very nice. Battery life is still TBD under normal use (as I was hitting it pretty hard with all the installation work), but it definitely goes for at least something like 6+ hours.

Think this one will work out well for some time into the future – the OS is supported until 2021!

OpenVPN on Tomato with Android and Linux Clients

I’ve been wanting to do this for a very long time. When away from home I sometimes need access to the systems (or data residing on those systems) back at home. I wanted to set up a secure means to access the machines behind my router’s firewall and one of the most versatile and secure ways to do that is with a Virtual Private Network (VPN). The problem was that this stuff is pretty complicated and even though the open source firmware we run on our router has had a VPN-enabled version available, I’ve been loathe to try implementing it.

Well, the garage control system project I was recently working on had a hardware failure such that I could not implement it in the original way intended (until I replace the CAI WebControl board central to it). The board failed in such a way that it would not accept PLC programming but would still respond through the default web interface – which unfortunately is not sufficiently secure to expose to the internet directly. However, we were going away for an extended period and I needed to be able to access it while away. A perfect application for VPN technology, I could keep the “vulnerable” system firewalled behind the router and poke a secure hole through it using the VPN to control it from afar when needed. Just the shove I needed to get going on the VPN!

Curiously enough, in googling, I was able to find various basic tutorials about setting up a Tomato VPN-enabled router (which is Linux based) as a VPN server with Windows clients and creating the certificates and keys on Windows but pretty much nothing simple about doing so with other platforms like mine – Android (again Linux based), Linux and Mac. The ones about setting up a VPN with Linux all seemed to want you doing everything down in the weeds of config files and installing VPN packages on your own server (not a router). Not what I wanted.

The good news for you and me is that I figured out how to get this done with minimal effort and it pretty much worked perfectly on the first try, so I’m writing it up here for future reference and to share with any others following this path. Looking back, it wasn’t that hard but the lack of clear guidance made it all confusing. All that said, here’s some clarity on how to get it done:

Creating Certificates and Keys

On Linux Mint LMDE (Debian Linux) workstation, using Synaptic or another package manager install:
openvpn
easy-rsa

This will install the easy-rsa scripts into
/usr/share/easy-rsa

Taking note of the instructions at http://openvpn.net/index.php/open-source/documentation/howto.html#pki, I did the following:

Copy the easy-rsa files to another location that will persist after package upgrades (note, this location already existed as a result of the openvpn installation and contained the single file update-resolv-conf, so maybe that claim is misleading?) and cd into that directory:
sudo cp -R /usr/share/easy-rsa/* /etc/openvpn/
cd /etc/openvpn

Edited the vars file using vi to set the KEY_COUNTRY, KEY_PROVINCE, KEY_CITY, KEY_ORG, and KEY_EMAIL parameters. These were at the bottom of the file for me. From what I can tell, the two email entries are identical but for the quote symbols. I presume the quoted one is meant to be the “real life name”, but nothing I could easily find via google confirmed or contradicted this – so I just set them both to the same address.

export KEY_COUNTRY="US"
export KEY_PROVINCE="MA"
export KEY_CITY="MyCity"
export KEY_ORG="My ORG Name"
export KEY_EMAIL="vpn_contact@myDomain.com"
export KEY_EMAIL=vpn_contact@myDomain.com

I then completed the rest of the steps at the above link using root/sudo priviledges, creating the certificate authority, server certificate and key and then the client certificates and keys. What I found online was not very informative on this point, but the Commmon Name (CN) must be entered each time you build these items and should be varied so as to be descriptive. So, for each command:

./build-ca
For this I specified my own name as the Common Name (I’m my own certificate authority) and it generated two files, ca.crt and ca.key (note, these are not named after the Common Name given, unlike the following).

./build-key-server ServerName
I gave my intended VPN server name as the ServerName which it then used as the Common Name and generated ServerName.crt, ServerName.csr, ServerName.key plus a 01.pem file and changed the index.txt and serial files in the keys directory.
NOTE: I also here encountered something different than that laid out at the above URL, for each key it asked me for:
Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:
which I simply clicked Enter through (I presume to set as blank, as no password was later asked for).

./build-key ClientNameHere
I gave unique descriptive names for each client and it created similar files to the server ones above, named per the client names I gave, created sequentially numbered pem files and updated the index.txt and serial files.

./build-dh
Returned Generating DH parameters, 1024 bit long safe prime, generator 2
This is going to take a long time
(But it didn’t – it was less than 5 seconds on my main Linux 64 bit workstation)

In the Arch Linux wiki entry for EasyRSA it stated that there was a need to convert the server certificate to an encrypted .p12 format for use on Android. I found this to not be needed, using the OpenVPN for Android client from the Google Play Store.

In order to provide additional TLS security and to protect against potential denial of service attacks against my router/VPN server I also set up an HMAC signature:
openvpn --genkey --secret ta.key

Setting the server and clients up…

As I created all the certificates, keys, etc. on my main Debian workstation, I needed to transfer those files to the associated machines. First I used my browser and the Tomato-powered router (VPN server) web interface to set up the VPN server following the info here *except* for using TUN instead of TAP. Installing Tomato is covered in my other blog post. Here’s screenshots of my settings (click on them to enlarge):
Tomato-Version
Tomato-VPN-Basic
Tomato-VPN-Advanced
Tomato-VPN-Keys
Tomato-VPN-Status

Connecting with a Linux machine. I then set up the test client on my Mint LMDE/Debian laptop following the leads at https://support.hidemyass.com/hc/en-us/articles/202721596-OpenVPN-Setup-with-Network-manager-on-Linux-Mint-Mate , which dragged along a bunch of other required packages including openvpn, easy-rsa, etc. I imported the certificates and keys when setting up the VPN connection using NetworkManager. Trying to connect via this initially failed. I thought this might be because I was on my home network at the time, so I proceeded to set up my phone as a client to see if I could use the cellular network to test outside access.

Connecting via Android. I installed OpenVPN for Android from the Google Play store onto my cell phone. Copied over the certs and keys to my phone using USB cable and set up the connection in the app. Took a bit of twiddling to figure out where everything went and which boxes to check, but it connected quickly once set up. Could access private resources behind the router firewall now! I went on to set up my Android tablet with the same app.

Connecting from an Outside Network. Brought my phone, Linux laptop and Android tablet for a drive to find an available Xfinity Wi-Fi connection. Tried each client to access the VPN once connected to some poor folks’ wireless access point (why folks stand for Comcast doing this, I don’t know), and they all connected quickly and could access my Garage Control System web interface on my home network… success!

Note that in none of these set-ups did I need to edit or create any configuration files manually on the clients or server, despite lots of other tutorials making great points of this! It appears each of the OpenVPN server and client implementations I used took care of this for me.

The only bit of weirdness is that I cannot figure how to directly disconnect from the VPN using NetworkManager under MATE desktop on my Linux laptop. I can disconnect the VPN by dropping the wireless connection overall. There should be a “Disconnect VPN” option within NetworkManager but I don’t see it on my laptop when I’m connected (it is there when I’m not!). But that’s a (minor) problem for another day.

I’ve found the disconnect option in the VPN menu under NetworkManager and that can be used to drop the VPN connection. The Android clients have a connection status entry in the notifications list which provides a disconnect option once clicked on. All good to go now!

Spicing Up Your Dance Collection with Pepperplate

In this post I will lay out some history of how I have managed my dance collection and what I am currently exploring as a way to greatly simplify the work of building and maintaining that collection and making it available whenever and wherever I am.

I’ve been calling (leading/prompting) contra dances for several years now. One of the things you need to figure out pretty quickly when you decide to become a caller is how you’re going to record and organize your dances.

Like most beginning callers, at first I recorded dances by hand on index cards (or all too frequently, any spare sheet of paper I could find at a dance). As my collection grew (and I started actually having to use these compositions to lead others through the dances) my requirements for standardization and legibility grew. (In all fairness, my background includes quite a bit of business process work – so I’m a bit of a process wonk.)

So for a couple of years now I’ve been working with a system which typically involves too much work. Why? Because I chose to standardize on using 3×5 inch index cards, which turns out to be pretty darn small. The 3x5s have enabled me to carry my core card collection easily in my dance bag if I want. But the small size means I need to really work on re-writing a lot of the material I gather to abbreviate or summarize and reduce to a standardized shorthand format I record on my cards. The real estate has been very constraining (but in fairness has made me really good at slimming down language clearly). And when someone else wants to see my card or I post a dance to a discussion group there’s sometimes questions about the notation.

I have been creating these cards using a template I had set up in Open Office Writer (now Libre Office) which then I would laser print 4-up on card stock as I added new ones or a given card wore out/was revised and then cut them with a paper trimmer to size. I could also export the cards to a large PDF file containing my whole collection. I kept both the pdf and original files backed up and synchronized across several computers via Spider Oak so I would never fear losing my collection. An example card:
A dance card format example.
My workflow for finding dances and transforming them into a usable card was essentially:

  1. Find a dance I liked. This could be from dancing or seeing one danced or based on something in email from a group/forum, etc.
  2. If got in person, I originally would scribble it down. Sometimes a caller would offer to email it to me. My latest trick has been to either take a cell phone picture of a caller’s card or quickly get the dance name and/or moves entered into Google Keep on my phone.
  3. If via email, I tag the email with a “Dance to Collect” tag in GMail which becomes a queue to transcribe from.
  4. Discover my dances in queue (Keep, email or photos) and review them for quality/suitability. Was I just in a dance trance and got carried away or is it really a good one? Will I actually call it? If all good, continue on. If not, delete or recycle the paper.
  5. Process worthy dances into standard format, adding them to the master Writer file. Queue them in the “dances to review” section and when there’s a suitable chunk, print on recycled regular sheets of paper to try out.
  6. Kitchen Validate. Try dancing my transcribed card in our kitchen. If needed, cajole other family members to run through it with me. Apply my now standard set of QA checks to the dance (progresses? work for both roles? etc.) and create teaching notes as required for when I’d call it.
  7. Dance Validate. When a suitable opportunity presents, call the dance. Note any key learnings on the card and mark it as validated as applicable. Factor in any dancer or musician feedback (often noting the tune chosen, if I’m sharp enough to ask).
  8. Update Cards. When I’d think of it, I would drag out my cards and scan them for ones with handwriting on them and record that information back into the electronic copy. If significant, I’d reprint the card(s).

As you can likely tell, that’s a lot of work. However, my cards enabled me to do a pretty good job calling even material that was new to me. I often got positive comments from musicians I worked with about the cards being very usable.

If you’re a caller, you might ask why I wasn’t using one of the existing caller tools to capture my cards, like Caller’s Companion or Dance Organizer? Well the answer is that I don’t have any iThings or WinThings. I run Linux on all my computers plus my cell phone and tablet are using Android currently. Sadly both of the established caller solutions don’t support any of what I’ve got.

So in a fit of frustration the other day I launched into yet another of my ~yearly reviews of the caller/leader software out there and found the dedicated applications landscape to have essentially remained unchanged. I thought briefly of setting up something on my own domain, veino.com, to do this as a database application but that would be limited to where I could get on a network. So, as an open source enthusiast, I started thinking creatively (we often need to do this, as popular “local app” tools are frequently omitted for Linux in particular). My breakthrough was thinking “what is a dance card?” and my answer was “it’s effectively a recipe for a dance.” With that insight, I started researching the recipe management software solutions out there. Again, I found a lot of stuff for OS X and Windows, even Android and a bit for linux. As I looked into it I realized my criteria basically boiled down to:

  1. Being able to add or edit dances anywhere I was on any device
  2. Being able to print them to hard copy if needed
  3. Being able to organize them into a program for an evening.

These were the core requirements, several ancillary ones flowed from there. These included the ability to classify dances in standard ways for filtering, searching to quickly find one, and managing my work queue. Also important was the ability to work offline when a web connection was not available (and sync that work when connected again).

The end result of my search was finding the Pepperplate recipe suite. It supports all my electronic devices, either through local apps or website tools. It supports tagging, filtering and search. The dance and meal analogy gets extended via treating a dance as a dish, a program as a menu and a booked event as a planned meal. It supports sections (parts) of a recipe, like sauce (A1), ingredients and instructions (moves and calls/teaching points). Pepperplate provides for adding dishes to a menu, and menus to a meal. I find that the analogy fits pretty well and I can use this tool to do most of what I want for my dance collection seamlessly. It also supports sharing recipes (dances) in a couple of easy ways.

The biggest difference from what I’ve otherwise found in the caller tools space is that this will work with pretty much all popular (and even unpopular) devices and that it automatically syncs across them. And not that it really matters given the relatively low cost of the existing dance leader applications, but it is also free.

I’m in the early stages with Pepperplate and tried calling from it for the first time just this past weekend. I only have a limited set of dances in the tool so far but it has been doing pretty well. I’m no longer severely space constrained! I do have some criticisms and have discovered some workarounds (mostly Android settings) to get around them. And BTW, there’s a big plus for me: the Android app includes a timer for each dish (dance) in a menu (program), so I can set it for how long I want to run the dance and a “can’t miss” message pops up to keep me on track.

In fairness, there are some risks and glitches with using Pepperplate for a dance collection beyond the obvious. These include a dependence upon a business with a not entirely clear how they make money business model. They might also not be happy with it being used this way (though from a quick review of their Terms of Service it appears to not be in violation and doing so just provides more eyeballs for their ads served). However, the data is stored locally on the device for off-line use and (at least on Android) is in a format that can be backed up and extracted/manipulated should Pepperplate.com go belly up.

Is it ideal? No, but it’s ~85-90% of the way IMO. Until something better comes along, I think I’ll be using Pepperplate to manage my dance collection going forward.

In a later post I’ll cover my experiences and tips with using the tool for this application: limitations I’ve found as a dance organizer (and even as a straight recipe) app, how I’ve set things up for ease of use/applicability and how I’ve fit Pepperplate into the dance collection workflow I lay out above. A quick preview: it has made things much easier!

Recovery of Files from a Unbootable VirtualBox VDI

I do most everything computer-wise with open source software, but the one hold out remaining that requires the use of a proprietary OS is TurboTax. As a result, TT ran in a Windows XP virtual machine under VirtualBox on my Linux desktop. Unfortunately, after completing our most recent return, I got a little excited to do some basic housekeeping and tried to merge snapshots from the VM in order to save some disk space. Unfortunately, as the attempt at merging snapshots resulted in an error being reported by VirtualBox that basically amounted to “you’re really screwed, buddy” but put in much geekier terms with a bits and bytes error code. A later attempt to re-merge or boot the VM again did not work. The virtual machine claimed that key windows files (like the kernel) were not available. Argh!

OK, so I’m usually pretty careful and save off critical files from the Windows VM to the Linux host. I sadly did not do that for the very-last-as-filed TurboTax working file (I had an interim copy from several hours earlier but I know we made changes later). Had the pdf copies of our returns but not the final version of the .tax2011 file, which normally copies over key details to our next year’s return. And of course, hadn’t yet set up SpiderOak to backup the files from within the VM to the cloud. Double argh!

As the VM would not boot, I tried various alternative boot scenarios to get at the files but none of them worked, using either a Windows install CD or a Linux live CD image within the VM. Furious Googling finally turned up a useful working solution to allow access the files on the Virtual Disk Image (VDI) associated with the VM. Was then able to copy out the files needed from within the virtual Windows environment to native Linux file storage. Phew, dodged that bullet! Here’s what I did under Linux Mint LMDE 64-bit to get access and then clean up afterwards:

Install Required Packages
Using Synaptic, installed the qemu-utils package, which dragged along a bunch of dependency packages.
bridge-utils (1.5-6)
ipxe-qemu (1.0.0+git-20120202.f6840ba-3)
libaio1 (0.3.109-4)
libiscsi1 (1.4.0-3)
libspice-server1 (0.12.4-0nocelt1)
libusbredirparser0 (0.4.3-2)
libvdeplug2 (2.3.2-4)
qemu-keymaps (1.1.2+dfsg-6a)
qemu-kvm (1.1.2+dfsg-6)
qemu-utils (1.1.2+dfsg-6a)
seabios (1.7.3-1)
sharutils (1:4.11.1-2)
vgabios (0.7a-3)

Gain Access to the Disk Image
Within a terminal window, executed the following commands:
lsmod | grep -i nbd
Nothing was returned, so the nbd module was not loaded already. Loaded it:
sudo modprobe nbd max_part=16
Run qemu-nbd to expose the entire unbootable image as a block device named /dev/nbd0, and the partitions within it as subdevices.
sudo qemu-nbd -c /dev/nbd0 WinXP_VirtualBox.vdi
The referenced blog posting/commentary said to issue a partprobe command, but I got an error about it not being available and didn’t seem to need it as the partitions were visible without it. Could see this by:
ls -l /dev/nbd*
To determine partition details:
sudo fdisk /dev/nbd0
and press p
This revealed the desired Windows NTFS partition from the virtual disk:
Disk /dev/nbd0: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders, total 20971520 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xdc94dc94

Device Boot Start End Blocks Id System
/dev/nbd0p1 * 63 20948759 10474348+ 7 HPFS/NTFS/exFAT

Access and Copy Off Files
OK, so create a mount point for the virtual disk and mount it READ ONLY:
cd /
sudo mkdir RECOVER
sudo mount -t ntfs -r /dev/nbd0p1 /RECOVER

Finally I could look at that mount point and recover the files:
cd /RECOVER/
cp -p /final/linux/resting/place/

Cleaning Up
Once I got all that I needed off the VDI, unmounted the image and shut down the qemu-nbd service:
sudo umount /RECOVER
sudo qemu-nbd -d /dev/nbd0

Then used Synaptic to remove all the qemu packages I’d just installed, to prevent the accretion of bloat hopefully never needed again. I’m trying to keep this Mint LMDE install tidy and avoid an OS reinstall for a good long time!