Kevin Parker Archive

One repository or many … the answer is neither

One of the most commonly asked questions these days is “Should all our source code be in one repository?” This is a complex question and leads to a somewhat interesting set of answers.

Before we get to that lets try and understand the question a little more and find out why customers asking this? In IT we like to centralize and optimize. Gathering all the code in one place is seen as the next logical set of distributed data ripe for centralization and optimization. All in one place means we can manage access better, manage backup and recovery better and ensure everyone is able to maximize the reuse of code.

However this flies in the face of modern developer behavior. At large and small IT organizations we see developers downloading open source source-code management systems for themselves and their teams. Instead of having one repository in one place we are seeing repositories on every server and developer hard drive creating a vast digital archipelago of repositories where processes and standards evolve on a team by team basis mimicking the finches on Galapagos recorded by Darwin.

And this is the dilemma. Corporate responsibility drives towards a single repository strategy but developer behavior wants local control and ownership of their code.

What does corporate want?

So what does corporate really want when they say they want a single repository? Typically they are trying to address multiple concerns and typically these are they:

  • Visibility into all the artifacts in the repository
  • Central access control over the artifacts
  • Conformance to governance guidelines and audit reporting requirements
  • Segmentation of the artifacts to match separation of duties mandates
  • Support for shared code and refactoring initiatives
  • Enterprise wide impact analysis
  • Control over misuse, misappropriation and malicious activities
  • Consistent backup of the repository

None of these are architectural in nature: they are all functional requirements that are easy to satisfy with a single repository and very difficult, impossible in some cases, to achieve with team-based repositories.

What do developers want?

Developers want the least amount of technology and process in order for them to develop at speed. To, as Mark Zuckerberg described it, “move fast and break things.” This means:

  • Solutions they can obtain without budgetary permission
  • A repository that is easy to use and flexible to their needs
  • Low process, governance and control
  • Easy (or no) administration
  • Simple (or no) licensing
  • Fast checkout and checkins (especially GetLatestVersion) across the LAN and WAN

Once again, these requirements are not architectural. They too are just a list of requirements. While they seem in conflict with what corporate governance demands there is common ground and a proper technical solution that meets both sets of requirements is possible.

Missing pieces

Developers fear having their code hosted on a platform that they are not developing for. Mainframe developers would never countenance their COBOL code hosted on Windows, no Unix developer would accept their code hosted there either. Developers in Beijing find it hard to accept their code hosted in Bulgaria and managed from Boston. Add to this the numerous code pages and, perhaps, ASCII to EBCDIC conversion issues that would ensue.

Most developers these days use code analysis tools designed for the development platform they are using so this means keeping the code on that platform and that in turn means duplicating the code from the single repository back to the distributed platforms.

As I said at the beginning this question raises many interesting issues. None is more pressing than this though.

Neither of these positions, single repository versus multiple distributed repositories, takes into account is that the source code repository represents the collected intellectual property of the corporation. It is a business’ most valuable asset, far beyond the goods and services they provide, and this is why it has become the single target and focus of hostile foreign governments, unscrupulous competitors, disgruntled employees and organized crime.

Secure SDLC: the next standard in repositories.

In tomorrow’s repository the design needs to represent best practices in secure data management. Protection of the repository is of utmost importance. This means that our repository must have:

  • Single point of access control
  • Robust auditing
  • Encryption of artifacts
  • Tamper detection of artifacts, logs, audit trails, reports and the software itself

The ideal repository architecture

What makes the ideal repository architecture is neither single nor multiple repositories.

Here are the key ingredients and, as you will see, they satisfy all the corporate and all the developer needs:

  • Secure repository defended against exfiltration and infiltration of code
  • Process centric allowing enforcement of one (or many) development processes irrespective of platform and in support of all development methodologies
  • Secure, immutable logging and audit trails
  • Single point of user and tool administration
  • Artifacts stored on the platform of choice by developers
  • Artifacts backed up by native utilities optimized for that platform
  • High speed performance over LAN and WAN
  • High speed performance irrespective of the user load, irrespective of the size of the repository and irrespective volume of versions and changes being managed and tracked
  • Caching of a minimal amount of code as needed reducing duplication and limiting misuse

We call this a Single Virtual Repository.

From a management and administration point of view it appears as a single repository but, behind the scenes, the SCCM software manages all the artifacts in their respective locations on their respective platforms.

From a developer’s point of view their code is collocated with the team allowing for the fastest possible access. It also means that code analysis tools are able to execute on the code natively without duplicating the code. Each team can have their own, or a mandated process, as processes and access rules can be defined at a project or even an artifact level.

Central control but distributed data.

Why Serena

The world’s most advanced and successful Software Change and Configuration Management (SCCM) solutions are both from Serena.

For mainframe developers it is the legendary ChangeMan ZMF whose “develop anywhere – deploy anywhere” approach allows development on the mainframe or in Eclipse-based IDEs for deployment and execution on any of the mainframe platforms from z/OS to z/Linux to Unix System Services and Websphere Application Server.

In the distributed space the unrivaled technology is Dimensions CM which is used by the most advanced and technically savvy organizations in the world in defense, intelligence, finance, insurance, pharma and many more. Its exceptional abilities provide developers with sophisticated tools that drive development velocity and code quality.

Both ChangeMan ZMF and Dimensions CM are designed around the Single Virtual Repository giving developers and corporate governance teams exactly what they need.

And for enterprises whose development efforts extend across the mainframe/distributed divide Serena Release Control brings a Single Virtual repository across the platforms coordinating development and release activities with a single point of view and control into ChangeMan ZMF, Dimensions CM and some third party repositories.

Serena has, for more than three decades, led the field in SCCM. Those years of experience supporting the world’s most highly regulated large enterprises in their most complex development tasks give us a commanding lead in the design and implementation of source code repositories.

Don’t be fooled into seemingly inexpensive solutions that create sprawl and drag as performance slows over time. But also don’t be seduced by the seeming simplicity of one repository to “rule them all.”

A single virtual repository is the only solution that meets all the needs of developers and IT governance and Serena solutions are the only ones that offer that and that are ready today for a world driven towards the Secure SDLC.

 

 



In the next few weeks Serena is going to be announcing important new versions of our ALM, Release and Deploy solutions.

Ever since Serena’s CEO, Greg Hughes, introduced the concept of “Move Fast Without Breaking Things” at our User Conference in Washington DC in February, we have seen an overwhelming acknowledgement from our customers and partners that this is the perfect encapsulation of what modern application development and deployment means to them.

For Highly Regulated Large Enterprises (we call them “HRLEs”) software is developed in the most demanding environments in the world. With a myriad of technologies, a dispersed workforce, new compliance and regulatory demands every day, time-to-market time-frames often counted in hours and with every methodology possible driving velocity and every tech innovation driving disruption these organizations face a critical challenge. How exactly can you “Move Fast Without Breaking Things?”

This summer we will see how that is achieved in the “Summer of Speed” launches of four of Serena’s flagship solutions.

In June we will have the first of these launches with our Release and Deploy solutions. These solutions are anchor-technologies for your DevOps infrastructure and are essential for your Continuous Delivery initiatives. With release frequencies accelerating and deployments stacking up on the threshold of production, sophisticated IT organizations, like those you find in the HRLEs, demand exceptional solutions to meet the challenges. Come back here for more information on Serena Release Control and Serena Deployment Automation.

In July the launch of our latest ALM (Application [Development] Lifecycle Management) solutions will be revealed. HRLEs need deep visibility into the activities of the development teams and developers need advanced tools to optimize their development efforts. This summer will see the fruit of the continuous innovation from the Serena development teams that will bring to life reality of a common platform for all the developers in the organization whether they are Agile or Waterfall or anything in between, whether they are developing for mobile or mainframe, for teams of 2 to 2,000, collocated or dispersed across 6 continents. Dimensions RM for requirements management and Dimensions CM for configuration management have long set the standard in ALM. These two solutions have been updated to bring new meaning to visualization as both a way to organize and manage the software delivery process. Watch this space for more information on Dimensions RM and Dimensions CM coming this summer.



JenkinsAs the leader in deployment automation, Serena is a proud sponsor of the Jenkins User Conference 2015 World Tour. Come visit us at our booth and meet our experts in DevOps, CI/CD and enterprise release management.

Network Learn Explore @ the largest gathering of Continuous Integration and Continuous Delivery experts on the planet.

Want to tap the collective knowledge of a vibrant community of CI/CD practitioners? How about being able to network with other Agile and DevOps practitioners, just like yourself, looking to learn what others are doing across the software delivery process?

You can still sign up for the conference and ensure your spot.

Register for just $399 (US) / £399 (UK).

You’ll notice some exciting changes this year, including:

  • Event expands to two days* – that’s double the number of sessions, double the number of networking opportunities.
  • Opening keynote by Kohsuke Kawaguchi – Kohsuke is the original founder and creator of Jenkins.
  • Special keynote by Gene Kim at US East and US West - Gene is a multiple award-winning CTO, researcher and author of The Visible Ops Handbook and The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win.
  • Ask the Experts returns* – with expanded hours and more opportunities to ask your most pressing questions.
  • The Jenkins User Conference will run concurrently with CD Summit* in each region, giving attendees an opportunity to peek into the DevOps/IT management side of continuous delivery.
  • The two conferences will share common activities* such as an integrated partner exhibit area, meals and an evening event. Additionally, attendees for one conference can cross-attend sessions at the other conference, making JUC 2015 a two-for-one event.

* at US East, Europe and US West.



FullSizeRenderSo we are off and running with xChange 2015. Record crowds in the breakout sessions and a packed house for the kickoff general session this morning.

Greg Hughes talked about the need to “Move Fast Without Breaking Things” and his first love … a 128k RAM, twin 320kb floppy, 32lb “portable” computer. He drove home the critical need of modern organizations to create an infrastructure environment that supports dev teams while, at the same time, ensuring control, visibility and compliance. Every organization is “facing unimagined challenges securing their software repositories” and he laid out a set of 5 best practices that can be actioned immediately.

In describing Serena’s product strategy each of Serena’s development leaders showed off the latest innovations in each of the product lines. With new releases of every Serena product in the last 12 months and two major product announcements today with the release Serena Service Manager 5.2 and Serena Deployment Automation 5.2 it was an important start for the conference. Dimensions RM showed off the first ever Requirements Visualization feature that lets users drag and drop requirements, dependencies and relationship right on to releases. Dimensions CM showed the ChangeSet Graph which lets you see development teams building and deploying code in real time. The ChangeMan ZMF team talked about their new, high-performance, migration tool designed to automatically eliminate the repository sprawl on the mainframe.

Keep an eye on Twitter and follow @serenasoftware and look out for #serenaxchange for the latest updates.

 

Tags: xChange


Well! We’re all set to go! We have a great agenda, a fantastic lineup of speakers and some fun activities planned. If you haven’t registered yet there is still time – you can register here.

Let’s take a quick tour of the highlights …

  • Over 70 in depth technical sessions – nearly half delivered by customer practitioners just like you
  • Major new product announcements you will only hear at xChange
  • Incredible end-to-end demo of modern software development infrastructure – “The Mother of all Demos”
  • Important new thought leadership about The Secure SDLC – new whitepaper published at xChange

Of course we’ll be celebrating too:

  • The 2015 xTravaganza will be a look back and celebration commemorating the 30th birthdays of PVCS, ChangeMan ZMF and Dimensions CM with special guest appearances of some of the thought leaders who created these technologies
  • The 2015 Innovation Awards (aka “The Douggies”) presented by Serena’s President and CEO Greg Hughes, will it be you?
    • Award for Innovation Excellence to the customer who has deployed a Serena Solution in a novel way
    • Award for Value Creation to the customer who has made the most dramatic savings for their organization
    • Award for Use Satisfaction to the customer who has changed the organization most impact-fully

Looking forward to seeing you there. Contact Victoria Tse today for special pricing deals.



If you are planning on coming to xChange I want to remind you that the early bird pricing expires on December 31st. Right now you can save $200 off the registration fee.

If you haven’t looked at the agenda yet I’d recommend hopping over to the xChange website so you can the amazing list of topics and keynotes we have planned for you.

See you in Washington DC in March!

Tags: xChange


The best just got better.

With the release of ChangeMan ZMF 8.1 a new era of software development becomes possible for the mainframe. With over 400 customer requested ideas implemented and groundbreaking innovations in cross-platform SDLC support, this release makes managing software development, from idea to deployment more streamlined, more automated and more reliable than ever.

Leading enterprises are under pressure to deliver innovation rapidly to satisfy their customers, while maintaining high quality and integrity, and reducing cost and risk. Agility and accelerated application delivery is required but companies struggle to deliver mainframe changes at the pace that the business demands. New Serena ChangeMan ZMF v8 capabilities enable mainframe application teams to deliver faster and at lower cost without compromising the enterprise scalability or security.

Here are some highlights:

  1. Release and Deploy Support For Eclipse and Windows Clients

Serena ChangeMan ZMF 8.1 provides development, release and deployment support for both Eclipse and Windows environments. Developers, release managers and business stakeholders can manage mainframe deployments and releases from a distributed client. Development teams can develop code and manage changes as they transition across environments.

  1. Support for High Level Language Customization Exits (HLLX)

Customer-specific business rules can be implemented in COBOL, REXX or any other high-level language. Pre- and post-exits are implemented at strategic points within the ChangeMan ZMF workflow and are implemented and executed across both mainframe and distributed clients.

“We tested the HLLX feature and were extremely impressed at the flexibility in which we could modify the behavior of ChangeMan ZMF. This allows us to leverage the development process as a clear competitive differentiator.”

  1. Global Application Administration Update

Simplified administration means setup and deployment of ChangeMan ZMF instances can be done in minutes ensuring dev and test teams always have the right resources when they need them

  1. Global Application Administration Update

Over 450 additional feature enhancements improving performance, security, usability, development and administration are included

As the most innovative release in more than a decade ChangeMan 8.1 sets the standard that all others can only aspire to. This is the ideal solution for mainframe teams that are under pressure to deliver high-quality, valuable software in an efficient, fast and reliable manner.

ChangeMan ZMF 8.1 highlights the continued investment that Serena Software is making in this strategic product used by hundreds of the world’s most important companies.

And there is more just around the corner. Watch out for next month’s release of ChangeMan 8.1 Client-Pack.

 



We are gearing up for our next global user conference, xChange15, to be held March 22-25 in Washington D.C. Like our past xChange conferences, this one is going to be all about helping our customers get the most value our of their Serena software investments, with presentations and workshops featuring the best thought leaders, technical experts, fellow customers and technology partners.

In three jam-packed days, we provide over 60 technical sessions on the products you are using today, like SBM, Dimensions CM and RM, Serena Release Manager, ChangeMan ZMF and more. We are working on the full agenda right now and will publish it as soon as available.

All available information about xChange is always on the xChange website at www.serena.com/xchange.

Discounted registration is available through the end of this calendar year, so take a look at your training and conference budget in 2014 and decide to spend it on attending xChange. You won’t be disappointed!

A few things to note for xChange15:

  • Our call for speakers is open now. Selected speakers receive free conference registration. Please see www.serena.com/xchange for details.
  • We are offering a selection of one-day intensive pre-conference training sessions on Sunday, March 22 for the amazing low add-on price of $199. Space is limited to 20 participants, so register now for xChange15 and add one of these important training courses during your registration.
  • We have a set of hotel rooms at the Ritz Carlton Tysons Corner set aside for xChange15 attendees at a preferred rate on a first-come, first-served basis. Links to hotel registration are included in the registration process.
  • Finally, we offered a discounted rate to customers joining us from outside North America, as we know travel costs for international attendees can be high. Please contact your Serena sales, services or support representative for additional information, or just send an email to xchange@serena.com.

I look forward to seeing you in Washington D.C. in March!

-Kevin

Tags: xChange


Tuesday

Like most jobs in life, preparation is the key to success. After getting to know the Serena Deployment Automation technology by working with the free version (download from the Serena website) for a few hours (see yesterday’s post) I decided it was time to try for real.

My application was a Library Management System I developed a while ago for a public library in the United Kingdom. Like most developers I like to have something familiar to play with when I am learning a new technology.

So I started by defining my application to Serena Deployment Automation (SDA). The truth is that the help system (which is very helpful) suggested I defined the Components first. Partly because I like to try and test things to their limits and partly because I like to do the unexpected.

Create the Application

To start I clicked on Management, then Application and then Create New Application. I gave the application a name and a description and I was done. Easy. But was it too easy?

Once my application was created it dropped me into the Environment definition page. I was expecting this because I had been through the tutorials and samples when I first downloaded the Appliance. Here is where we define the target environments for the application. Every application lives somewhere. The environments are definitions of the locations you will be deploying to for development, testing and production. Each environment can comprise of one or more targets.

I clicked on Add Environment and the drop down menu invited me to pick from DEV, INT or QA. Well they didn’t suit me so I realized I needed to create my own Environments.

Create the Environments

So now my plan was off track and that made this whole thing even more fun.

I clicked on Environments and there were DEV, INT and QA. So I clicked on Create Environment and all I had to do was to give the Environment a name and description. Next I was shown the Environment Details page. It had no details of course because it had just been created.

The Application was yet to be associated with the Environment and it had no resources. It was then it dawned on me. My application comprises of three parts. The database, running all the time, the programs running when invoked and the scripts that run once each time the application is refreshed. These resources, these components could go to any of the servers in my environment. I needed to define these components so I could tell SDA which components go where.

So I should have followed the instructions after all and defined the Components first. Good to know the help system has my best interests at heart. Even though I went down the wrong path all the entries I made are going to be used when we get down to the deployment itself.

Create the Components

So I now click on Components and the Create Component button.

Here I am invited, as usual, to give my Components a name and Description and, in addition, details of their location and the repository type. SDA supports almost 20 different types of repository including PVCS, Dimensions and Subversion.

Summary

My foundation is in place. Now I have to fill out a little more of the details and decide how the deployment should go. This whole process took no more than 5 minutes. In that time I had set up the Environments, the Application and the Components.

What I really want to do is deploy my Application and its Components through the sequence of Environments I have set up. To do that we need to define the process we want the deployment to follow. And that is what we’ll do tomorrow.



 

A couple of weeks ago I wrote about downloading the new Serena Deployment Automation Appliance. This is the free, community edition of the exceptionally advanced Deployment Automation technology we introduced last year. You can get your own copy, free forever, at the community edition website.

The story so far

Since that post I have been working with the Appliance learning how to automate deployments. For about half an hour each day, for the past week, I have been pressing buttons, dragging and dropping and generally putting the technology through its paces partly to improve my understanding of how it all works but mostly to see just how much better automation is than the manual processes I used to use. I have to say I’m impressed! Let me take you on my journey and share with you how I became an automation-maven in just a week. In order to make this digestible I am going to write it in 5 separate postings.

Today, we have naming of parts* (Monday)

Serena Deployment Automation divides the aspects of deployment into 3 units of deployment:

  • Applications – this is the entirety of what you are deploying. It might consist of scripts, executables, images, configurations, in fact anything you need to upgrade your application from its current state to its upgraded state.
    • In my case my Application is the Llareggub (a fictional place in Wales) Public Library
  • Components – these are the distinct collections of items, often from specific repositories Dimensions CM, PVCS/VM, VSS, Subversion etc), that are going to be deployed. Each collection will be of similar types of artifacts that need the same deployment process.
    • In my case I had three collections of components: database schema changes in the /SQL folder, new programs in the /CBL folder and set up scripts in the /BMS folder.
  • Environments – are where the deployments are going to to go from and to. These can be single or multiple deployment targets such as a single server or virtualized shopfront like Amazon.
    • In my case I just followed the paradigm that was in use in the Appliance already of DEV, INT and QA, development, integration test and QA testing.

Each of these has associated attributes, the most important are:

  • Processes – these are associated with Applications and Components. The component processes allow you to create activities that are different for different classes of components and to create different process for those components. For example a set of database DDL needs very different treatment to a bunch of DLL’s. But how we apply DDL to a MS/SQL server is very different to how we apply it to an IBM/UDB server. Imagine these as micro-processes, as your toolkit for deploying this kind of component. Application processes are macro processes made up of a collection of the micro-, the component-processes. Here you can create a deployment process that initially loads the application on to a new server or a process for putting out a security patch.
    • In my case I had processes for deploying the application from DEV to INT and from INT to QA comprising of the stop-backup-apply-ddl-restart of the database, backup-deploy of the code and backup-deploy-execute of the scripts
  • Properties – describe the nature of the Applications, Components and Environments. They specify, for example, the source repository type, the identity of a deployment target, the approvers and many other attributes.
    • In my case I kept things simple and took defaults whenever I could.

Tomorrow – setting up the Application and the Components and a deeper dive into Properties and Processes. * apologies to Henry Reed, Naming of Parts and E. V. Milner, Baking of Tarts