Current at: http://www.turing.org/jon/resume/
Jon Gefaell
(408) 666-7983
Charlottesville, VA
jon@Turing.ORG
Summary of Qualifications
Expert in Unix Networking and Systems Administration with a long-term and comprehensive experience in the design, implementation, and administration of technical operations environments. I have extensive experience with highly available, mission-critical, public-facing sites. I am also proficient with secure practices, email, DNS, server provisioning, configuration management, monitoring, capacity planning, change control, business continuity, and other aspects of technical operations. I recently took a hiatus from my engineering career to create a small company focused on artistic creations. Running all aspects of a small company allowed me to grow personally and professionally while leveraging my engineering experience. Having achieved this, I am ready to return to the full-time engineering market.
Employment History
Principal - Operations, Customer Support, Training
Medical Data Networks L.L.C. (T1Pal) - Charlottesville, VA August 2021 - Current
T1Pal provides Software as a Service to the Diabetes community. We provide Nightscout and the iOS Loop app to a growing number of people who use insulin. I am solely responsible for all aspects of daily operations including customer support, training, mobile app release management, sales, marketing, and billing.
Owner/Artist
Notta Design L.L.C. - Charlottesville, VA 2018 - 2021
I started a very small company on a shoestring budget and formed a team to design and build a luxury custom electric torch ('Flashlight'). Essentially a sort of jewelry. I personally supervised and directed all aspects and was able to bring an innovative, very high-quality product to market. This required learning many new technologies and developing processes to design, machine, finish and assemble completed products.
Senior Unix Systems Administrator
Vindicia, Inc - Redwood City, CA Sept 2015 - Feb 2018
I joined Vindicia to create architecture for a new data center buildout and improve upon Vindicia's mission-critical PCI compliant technical operations environment
Improvements to the existing operational environments include:
- Automated configuration and change management tasks to be performed on hosts across the environment in a safe, reviewable, repeatable, and rapid manner. These included timely patching, PCI audits, deployment of corrected and standard configurations, etc.
- Surveyed host configurations to find a wide variety of hand-placed configurations to remove any unique and 'irreplaceable' systems.
- Created and distributed new standardized configurations and placed them under source control for tracking and documentation through change management.
- Identified key areas which would be used as the cornerstone of the new data center design and operation.
Built an integrated proof of concept ("iPOC") lab to model effective, integrated solutions. The 'iPOC' environment includes:
- Network-based host build, IP configuration, and configuration management
- Internal and external DNS with all hosts and addresses accounted for, service aliases, discrete subdomains, and standard format zone files, PTR records, and multiple third-party providers.
- Management of Hardware/IP addresses and asset information.
- Comprehensive 'Lights Out' remote operation from host racking through repair, maintenance, and replacement.
Elements from the iPOC environment made their way into all operational environments:
- Created internal, managed yum repositories for host builds, upgrades, security patching, third party, and internally developed platform software.
- implemented sophisticated remote TCP syslog with dynamically created directories and logfiles for local and remote application logging of system and local product applications.
Most significant achievement
- Developed automated host build and configuration management process for application and infrastructure hosts and successfully migrated entire technical operations, development and QA stacks from CentOS 6 legacy to CentOS 7
Linux/Unix Escalations Engineer
Verizon/Terremark - Cary, NC March 2013 - Nov 2014
I provided senior level escalation support for Unix systems. This primarily included return to service activities that could not be resolved by prior support levels. I also handled a variety of escalations for related issues with DNS, network, storage & backup. A focus was placed on identifying and eliminating recurring issues as well as training and mentoring junior level support staff. This included working together to resolve issues as an alternative to escalation. The results were incident reduction, improved MTTR and improved 'first touch' resolution.
Return to service tasks consumed no more than 50% of work time, often considerably less. Other regular tasks included creating and maintaining standard operating procedure documentation, detailed change instructions and requests, detailed post incident root cause analysis. Provided Unix SME guidance and participatio in phone bridges during high impact/severity issues with executive visibility. Worked with account teams (Service Managers, Account Customer Engineers) directly facing customers to resolve problems and suggest and support migrations and upgrades.
Senior Systems Engineer
Phoenix 2.0 - San Francisco, CA. May 2012 - March 2013
I built out RHEL Kickstart, improved DNS, reviewed existing product deployment process and configuration management scheme. Provided support and improvements for these processes and other as needed. Also conducted technical interviews for Phoenix consultant candidates.
Technical Operations Manager
Adobe Echosign (Formerly Echosign, Inc) - Palo Alto, CA. March 2011 - May 2012
I built the Technical Operations team and structure from the ground up prior to the acquisition of "EchoSign" by Adobe. I lead an operations team consisting of 3 engineers and supervised ongoing work by an outside consultant DBA firm. My team supported and grew a technology stack with Java/Apache/AMQ/MySQL running across controlled Preview, Stage & Prod environments in two data centers.
- Established procedures for safe, predictable and reliable operations such as change control, run books, deployment and theory of operations documents.
- Established Operations as site deployment owner, achieved zero downtime releases by staging database changes and other careful planning.
- Developed new Disaster Recovery/BCP plans.
- Improved data storage, archival and backup methods involving encryption and on/off-site storage.
- Improved network security through comprehensive audit and mitigation.
- Dramatically reduced site downtime and established 99.99% availability
- Introduced methods for secure use of sudo, ssh, and the ownership and permissions of users and processes.
Technical Operations Architect, Operations Manager
Panasonic Cloud Entertainment Company, Unit of Panasonic North America - Cupertino, CA. October 2010 - March 2011
- Managed outsourced systems admin team which had built out the prototype data center and began to introduce necessary changes to move forward to production status
- Developed Kickstart & RPM Packaging to rapidly provision and update applications allowing ~15 minute automated buildout of core systems including database
- Organized a unified working group between Operations, Development, and QA to share methods, tools, and skills
- Successfully demonstrated above with end-to-end OOB device registration and 3g account activation following rapid buildout
Infrastructure Architect, Operations Manager
Philotic/Causes.COM - Berkeley, CA. March 2009 - October 2010
- A very small company with 30MM+ Monthly Active Users) Causes is one of Facebook's leading applications and helps organizations raise awareness and funds for charitable causes. I took responsibility for all aspects of the company's technical operations. I launched an effort to achieve order and establish best practices in a challenging environment. I was able to drastically reduce outages and measurably improved reliability, availability and serviceability through several strategic and tactical initiatives.
- Negotiated specifications and contract for new Datacenter & IP Transit
- Managed equipment refresh including servers, network, power, etc
- Migrated services to new public IP block, new load balancer configs
- Built infrastructure including kickstart, puppet, nagios/cacti & snmp
- Utilized DNS/DHCP/IPAM in datacenter to great effect
- Achieved extremely agile, certain & safe provisioning and management of servers
- Built reliable and scalable mail cluster to handle 2Bn deliveries a year
Senior Unix Systems Engineer
Yahoo! Inc. - Sunnyvale, CA. March 2007 - February 2009
Member of a small team of Systems Administrators responsible for Yahoo's global UNIX and DNS systems Administration for Yahoo! worldwide.
- Responsible for operating systems provisioning, operational troubleshooting, datacenter tools and services infrastructure.
- Servers are a mix of FreeBSD and RHEL GNU/Linux on various x86 platforms from small to very large.
- Specialized in Global Traffic Management and general troubleshooting.
Global Infrastructure Operations (Consultant)
Network Appliance, Inc. - Sunnyvale, CA. March 2006 - March 2007
Reported to Senior Manager of Global Infrastructure
- Worked to understand (document) and improve IT infrastructure in areas such as NOC, systems availability and performance monitoring, directory services, email, secure remote access, systems enrollment, OS and Applications provisioning, patching and Maintenance.
- Review and Recommend new processes and toolsets for comprehensive ITIL integration.
- Implementation Lead for IBM/Tivoli, Opsware and other key initiatives.
- Member of Change Advisory Board. Reviewed and approved Enterprise Change Management requests and workflow.
Senior UNIX and Network Systems Engineer (Datacenter Architect)
Netblue, Inc. - Mountain View, CA. May 2004 - January 2006
Sole systems engineer for rapidly growing internet advertising company
- Started with existing datacenter operations in 4 ISP Cabinets with ~25 hosts running mostly Redhat 7.2.
- Enhanced security of operations with the wide implementation of ssh/scp.
- Introduced sudo and other methods to restrict and audit access.
- Created an automated NOC utilizing redundant servers with Nagios, Perfparse, Cacti, NTOP, Syslog-NG & Logwatch for remote logging & analysis.
- Host and services are provisioned through Kickstart and other automated means.
- Implemented and Utilized Trac Wiki for Change Management, and SVN for source management.
- Automated backups of infrastructure systems using rsync over ssh.
- Implemented network and server security with bastion hosts, firewalled network access through host IPTables and Cisco network gear.
- Provisioned HDS SAN, ISCI equipment & various hardware and software RAID schemes.
- Programmed Cisco WebNS for load balancing with persistence & redundancy for ~50 websites and other services.
- Provided on-call 24/7 NOC support.
- Migrated main operations to a new data center with 6 custom racks in a cage. At the time of my departure, there were over 100 CentOS GNU/Linux servers in three locations and three additional Systems Engineers who provide data center services which advance key business projects.
Senior Consulting Engineer
Apple Computer - Atlanta, GA. February 2002 - February 2003
Member of a team providing an overlay of Enterprise UNIX server expertise to Apple's Educational Sales Field Organization.
- Developed and provided training for Apple's systems engineers and account executives to help them understand and sell Unix servers.
- Worked with sales teams to identify and pursue significant opportunities.
- Developed and delivered presentations and demonstrations to key customers achieving significant success in demonstrating enterprise capability for Apple's UNIX server products.
Senior Systems Engineer
Napster, Inc. - Redwood City, CA. January 2001 - January 2002
Senior member of Team responsible for design and operations of the server farm.
- Created reliable, high performance web server farm.
- Supported large server farm with over 1.7M concurrent users sharing over 280M files
- Daily operational support of infrastructure (nfs, dns, dhcp, etc...)
- Linux systems for website(s), napster servers, various infrastructure.
Staff Engineer
Sun Microsystems, Inc. - Mountain View, CA. February 2000 - December 2000
Converted from contract to regular position. Accepted responsibility as Staff Engineer for all aspects of partner software engineering for Solstice Backup (SBU) product including sustaining product support and new product development.
- Successfully completed entire major product release lifecycle (front to end) with the release of SBU version 6.0.
- Participated as member of W-team to coordinate Product development, Customer support, manufacturing and marketing efforts.
- Authored product Engineering plans, Marketing Requirements Definitions,
- Responsible for all bugs and engineering escalations.
- Directed other engineers.
Contract Positions
Sun Microsystems, Inc. - Mountain View, CA. February 1996 - February 2000
Lead Engineer. Responsible for management of escalated customer support issues. Developed workarounds and product fixes for problems without existing resolutions. Managed Information necessary to coordinate Product development and Customer support efforts.
- Quality Assurance & System Test Group.Responsible for the testing of Fibre Channel & SCSI hardware RAID devices and software including drivers, GUI management tools, RAID Management software and Veritas Volume Manager/File System. Authored & Developed Test Plans. Automated procedures through jumpstart and rc scripts. Managed group lab.
- Lead Quality Engineer to perform system software and hardware testing for UltraSparc/AX system board for OEMs. Developed and implemented Test Plans providing 'go-nogo' decisions for product development and marketing.
- Provided Internet software and hardware expertise to team developing Internet software and HTML/CGI based administration tools. Ported and built various packages to be integrated to products. Managed project lab. Supported Sustaining Engineering for previous product version.
Senior Systems Engineer
Healtheon, Inc. - Palo Alto, CA. July 1996 - December 1996
- Responsible for initial installation and administration of highly secure and available web site, corporate and development environments
- Installed and configured Gauntlet firewall, Split DNS, NIS with Automount, ISDN and remote access using PortMaster, RADIUS, and SecurID token cards.
- Procured DLT/Legato tape system for the database (Informix), UNIX, and Windows workstations backups and archival.
- Worked with Network Engineer to plan and implement network topology for complex multi-server facility utilizing multi-homed hosts, Cisco routers, and high-speed ethernet switches. Renumbered internal networks and migrated from ISDN to T1 Internet connection.
Co-Founder & Chief Architect
Cornerstone Networks - Charlottesville, VA. July 1995 - December 1996
Co-Founded Internet Service Provider business.
- Designed and implemented a complete system including T1 IP connection, dial-in lines, dedicated client networks, and an accounting and billing interface.
Full services include DNS, Web, FTP, and Mail services (with Virtual Domain support) for a mixed consumer and commercial clientele.
Equipment included Cisco routers, USR modems, Sparc/SunOS, and Pentium/Linux.
- The business was subsequently sold to a telephone company.
Computer Systems Engineer
University of Virginia - Charlottesville, VA. January 1985 - November 1995
Responsible for data communications for Administrative Computing Services
- 1994 - 1995 Designed and implemented community network for county and city government. The project provides Internet access and presence for regional non-profit and government service organizations as well as numerous public use facilities in the regional library system.
- 1993 - 1994 The Hopper project. A public access Internet experiment. (see below; Turing Project)
- 1993 - 1993 Special Project, Multicast IP Experiments. Created multicast router and application platform for real-time audio/video experiments over MBONE meta-network.
- 1992 - 1994 Netware Lead Support Engineer; led a group of 6 engineers and technicians in the management and support of the campus-wide 180 server internetwork. Directly responsible for 24 servers within computer center facilities. Upgraded many systems to 3.x.
- 1991 - 1992 Designed and implemented statewide network for Continuing Education Division. Over 250 stations at 7 regional centers were connected with Proteon routers and 56Kbps links and then connected to the Internet at UVA. The Centers used SNA, Netware, and IP protocols.
- 1990 - 1991 Installed TCP/IP on IBM SYS/370 to provide FTP and TN3270 services.
- 1989 - 1990 Designed, implemented, and managed 80 station Token Ring network with Netware file and print servers, 3270 gateway, application programs (Wordperfect, Lotus, Dbase, etc?)
- 1988 - 1991 Continued responsibility for rapidly growing VTAM/SNA network (Now 1250+ nodes).
- 1987 Began migration to PC-based 3270 emulation products.
- 1986 Responsible for large (750+ node) VTAM/SNA network with local and remote 3270 terminals and printers.
- 1985 Responsible for processing daily jobs, input, and output. Designed and implemented a new online system for job submission and review, still in use as of 2002.