Dev Ops - architecture (local not cloud)

[Patrick Flaherty]

Most of our devs do their dev work on their desktops. When 1tb sata drives
are 600 bucks it made more sense to let devs have "perishable" work
environments. We're moving some of our compile/testing/deploy to jenkins,
but that's really just a pilot program. I'm playing with docker/vagrent as
well for testing environments.

That being said, what are your mount options for home directories? Sounds
an awful lot like you aren't async.


On Fri, Dec 6, 2013 at 9:31 AM, Greg Rundlett (freephile) <
greg at freephile.com> wrote:

> Performance comparison:
> svn checkout single repository on old infrastructure
> real    5m44.100s
> user    0m36.957s
> sys     0m14.757s
>
> svn checkout single repository on new infrastructure, but only using NFS
> for "read" (local working copy stored on local disk)
> real    3m15.057s
> user    1m18.195s
> sys     0m53.796s
>
> svn checkout same repository on new infrastructure, with writes stored on
> NFS volume
> real    28m53.220s
> user    1m45.713s
> sys     3m26.948s
>
>
> Greg Rundlett
>
>
> On Fri, Dec 6, 2013 at 8:35 AM, Greg Rundlett (freephile) <
> greg at freephile.com> wrote:
>
>> We are replacing a monolithic software development IT infrastructure
>> where source code control, development and compiling all take place on a
>> single machine with something more manageable, scalable, redundant etc.
>>  The goal is to provide more enterprise features like manageability,
>> scalability with failover and disaster recovery.
>>
>> Let's call these architectures System A and System B.  System A is
>> "monolithic" because everything is literally housed and managed on a single
>> hardware platform.  System B is modular and virtualized, but still running
>> in a traditional IT environment (aka not in the cloud).  The problem is
>> that the new system does not come close to the old system in performance.
>>  I think it's pretty obvious why it's not performing: user home directories
>> (where developers compile) should not be NFS mounted. [1]  The source
>> repositories themselves should also not be stored on a NAS.
>>
>> What does your (software development) IT infrastructure look like?
>>
>> One of the specific problems that prompted this re-architecture was disk
>> space.  Not the repository per se, but with 100+ developers each having one
>> or more checkouts of the repos (home directories), we have maxed out a
>> 4.5TB volume.
>>
>> More specifically, here is what we have:
>> system A (old system)
>> single host
>> standard Unix user accounts
>> svn server using file:/// RA protocol
>> 4.5TB local disk storage (maxed out)
>> NFS mounted NAS for "tools" - e.g. Windriver Linux for compiling our OS
>>
>> system B (new system)
>> series of hosts managed by VMWare ESX 5.1 (version control host + build
>> servers connected via 10GB link to EMC VNXe NAS for home directories and
>> tools and source repos
>> standard Unix user accounts controlled by NIS server (adds manageability
>> across domain)
>> svn server using http:/// RA protocol (adds repository access control
>> and management)
>> NFS mounted NAS for "tools", the repositories, the home directories
>>
>> Notes:
>> The repos we're dealing with are multiple "large" repositories eg. 2GB
>> 43,203 files, 2,066 directories.
>> We're dealing with 100+ users
>>
>>
>>
>> [1]
>> http://publib.boulder.ibm.com/infocenter/pseries/v5r3/index.jsp?topic=/com.ibm.aix.prftungd/doc/prftungd/misuses_nfs_perf.htm
>>
>> Greg Rundlett
>>
>
>
> _______________________________________________
> gnhlug-discuss mailing list
> gnhlug-discuss at mail.gnhlug.org
> http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
>
>


-- 
* Patrick **Flaherty  *|
* w:* 978 983 6597      *e:* patrick.flaherty at weather.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.gnhlug.org/mailman/private/gnhlug-discuss/attachments/20131206/8271d2d6/attachment.html