DVCS Asset Lock

First release September 2016

Warning:

This is not an asset published through Unity Asset Store. This is a software built for internal use that we are inviting other teams to on individual basis. While unity and others have created competing solutions in the meantime, this is the only solution that allows this kind of safety on local servers, without paying for cloud services. If you are interested in participating, contact us.

Overview:

Working with source control in a game development team has many complications compared to a regular software team. I wrote many articles on this topic (see website section), and they are meant as ideas and guidelines to ease this work. Here is the summary of typical setups I have encountered, related to source control.

Using a centralized version control system, such as Perforce. The need to be constantly connected to the server, inability to enforce its main purpose in Unity (forbid asset change) and inconvenience when working with source code assets make this choice less appealing than in years past.
Using a DVCS system (such as Hg or git) for everything, and having artists submit their changes with pull requests. This is much better, but not a complete solution. It creates lots of issues if there is not 100% clear separation of responsibilities between members. It requires for everyone to know at all times what everyone else is working on. In case it's not 100% clear who is working on what, it can create holdups and duplicated work. It is also less than ideal when working with Scenes and Prefabs, and forbidding of touching these my multiple people at the same time can create holdups.
Using a DVCS for code and CVCS (such as SVN) for assets. This has been used on big projects by co-located big studios. It has some benefits, it seemingly provides the best of both worlds. However, it impacts the DVCS system in a sense that it is not "consistent", the changesets from the 2 revision control systems will not match and any reverts through time will not be consistent. Furthermore, it doesn't solve the issue with Scenes and Prefabs, and it relies on complex include/exclude files for both VC systems. Lastly, since assets and scripts are linked in Unity (through inspector) any users wishing to integrate changes must run commands of both systems and be aware of how they interact.
Using DVCS for code and assets, and denoting the lock status of binary files with "tokens", such as cards with the name of the asset. This is close to being convenient, but it doesn't really work with remote teams, since external electronic tools of this type quickly become a chore and users start skipping it. Finding and contacting the holder of the token becomes problematic, both locally and remotely. Doesn't really have special cases for Scenes and Prefabs, so the waiting for editing these may create a holdup. Tokens may not get returned (or not returned immediately) on completion as editors naturally prefer to hold the rights to edit the asset, and they are mostly unaware of other users wanting the token.
Using DVCS for code and assets, and shouting the status of the lock for non-text assets. This is an actual technique used by at least one small team at game jams. Has no benefits over the previous one, and obvious drawbacks.
Using DVCS for everything, and buying an external Asset in the Asset Store that is specifically designed to help in merging assets. This makes the procedure possible (as opposed to outright impossible with existing tools such as Unity YAML) but is still not pleasant. Further, the merge itself is never going to be convenient - it can be less painful but not convenient. Since the merge is required to complete the commit, oftentimes the merging duty can come on a team member who does not have the necessary background to complete the merge. There is no mechanism to prevent the merge in the first place, and there is no clear explanation why the changes on a specific asset were made, so they may be properly approved or rejected. It does nothing for binary assets, such as sound files, models or textures.
Using DVCS for everything, and ignoring the issue. No bueno.
Using an archaic CVCS for everything, such as SVN. No bueno.
Not sharing source control and having one person as an integration hub. No bueno if there are more than 2 members.

The Proposition:

So, what are the solutions to these issues? More than a year ago I wrote an article about a hypothetical DVCS that would allow for file locking on a global level. And while the idea wasn't great in retrospect, there was something to it.

Later I have for my own needs developed a set of procedures, and a code base to solve this issue. I have named this solution DVCS Asset Lock or DAL. First and foremost, it's a procedure and a set of conventions, that have been made trivial to operate with through provided Unity Engine Extensions. Second, it integrates with existing DVCS (currently Hg only) to reason about the status of files locking and automate steps. Third, it does not prevent the users from breaking the procedure in any way. If the asset is considered locked, nothing can prevent the user from actually changing that asset in true DVCS fashion. However, if such breaches happen, the procedure would regulate it at the time of changes integration (more below).

How does DAL work?

DAL presumes the team uses ES Flow with source control, a term and process I have coined and published on my site, which is based on gitHubFlow. I will list it here for convenience:

Our workflow has the following assumptions:

All team members fall under 2 groups: Repo Administrators and Team Members. Administers are the ones that have full permissions on source code repository and have enough technical knowledge to do code merges. Administrators also have authority over which changes are going into production, including art assets.
There is only 1 'permanent' branch at all times. It is called default. All other branches are spawned from it, and are considered temporary. Whatever is on default branch is considered stable and ready for building.

Workflow goes like this:

All development starts in default branch. Before any changes are made by developers, they create a short-lived named branch that depicts the intent of the change.
Developers make commits and work on that branch. If they are stuck or need information, they communicate with our standard protocols (instead of Pull Requests in gitHubFlow).
Once the development on that branch is considered complete, pull request is open and Administrators review the changes. If the changes are accepted, Administrators integrate the changes into 'default' branch.
Other developers pull all history and always start work from default branch.
Development changesets are not rebased unless there is a specific need to do it.
When certain 'releases' (milestones) are reached, they are marked with tags. The releases would include: Conference Build, Publisher Demo, tagged version v1.2 etc.
If any specific hotfixes are needed off of a release, they are made by just continuing work from the tagged commit (anonymous branch in Hg). This ensures that it's clear for which version the hotfix is for. These hotfixes are deployed, and then merged into the default branch (not rebased).

So, once this procedure is in place, the team members do all the work required inside Unity. They do so with Right click menu on the asset that they wish to operate on.

The fist 3 options are the part of the asset. There are a couple of components to it:

Edit scene option is available only for scene type files and it creates a new scene for additive integration by following a set of conventions. Within this new 'partial scene' changes are safe to make and Repo Administrator manually reviews these changes and integrates them into the original scenes at the time of integrating a pull request. This procedure is supported by a set of conventions, so that it's always clear what needs to be integrated, when and why. The integration is done centrally (by integrating a pull request), but it typically takes very little time. Different users may edit the same scene in parallel and the integrator decides what goes in if the overlap happens. There is never any file conflict since originals are not modified.
'Create Variation' is available only for prefabs and scriptable objects. It relies on simpler conventions similar to the above noted, and also must be integrated manually by Repo Administrators at the time of integration.
Asset lock, the first menu shown above (and the majority of the code of the system) has several options, and is used primarily for binary assets, which cannot be merged. It is intended to provide pseudo-lock (a soft lock) and a convenient set of operations around it. It requires a web server that will host several provided PHP files and a MySQL database. This server can be an existing web server and must be available at all times in order to perform or query locking status. It could potentially also be the same server that has the repository hosted, but that is not required. If the server is not on for any reason, all operations except this section can proceed. Before running any of these operations users must fill in their data in a separate form. This includes the name and an email, the OS and a couple of other things. This setup will play nicely with multiple projects by the same team without additional setup.
'Check if locked' will simply perform a check whether the asset is locked for the asking user, on the asking changeset. All replies to these queries are shown as dialogue boxes. The operations are non-blocking for the editor, and the option will consult previously mentioned PHP files and interpret the responses. The message displayed will show when and by whom the asset has been locked, if it is locked. All other operations described below will properly check for locked status before continuing, but for user orientation it is recommended to check status first. The time of locking will be displayed in the user's time zone, which is pulled from the Operating System's time zone and conveniently converted for the user. Another possible type of message is that the asset is locked, and the user that is doing the check is the holder of the lock. The final possibility is the asset is not locked.
If the asset is not locked, then user may proceed further. They again Right-Clicks on the asset and choose E.S Tools - Asset Lock > Attempt Lock. This will fire a request for locking the asset, and is tied to the specific commit the user is currently on. It is NOT allowed to lock the assets on the default development branch. All locking must be done on the named branches, created by the user before the lock. This procedure will check for this condition, and will not allow operation if it is not satisfied. Lastly, the locking request will be sent to the database.
If the asset was locked, and the user needs the asset urgently this system has built in notification mechanism. User chooses the option E.S. Tools - Asset Lock > Ask For Unlock, and the system displays the confirmation dialog. If the user confirms, the email will be sent to the locker, with content similar to the following:

This email was automatically generated by DVCS Asset Lock Unity Extension by Ennoble Studios.

Dear Gru,

User Gru has asked for unlock of the asset at path:
'Assets/Tools/Editor/AssetLock/GetAssetPath.cs'.
The project in question is Project.

Cheers,
Your DVCS Admin Bot.

If the locker changes their mind about the asset just locked, they can use the option E.S. Tools - Asset Lock > Revoke Lock. This is possible only if the user is the actual locker, and if they are currently on the same commit where the locking happened. The system will check the validity of the situation and do the unlocking if conditions are satisfied.
System also has the reservation mechanism. It works as follows: user chooses E.S. Tools - Asset Lock > Reserve Asset option, and they will get a confirm dialog, and if they choose to accept it they will reserve a spot to be notified when the asset is merged back into the mainlane. This happens automatically when Administrator announces unlocks, and they do that after the merge has completed. This creates an automated email for now-unlocked assets with contents similar to this:

This email was automatically generated by DVCS Asset Lock Unity Extension by Ennoble Studios.

Dear Gru,

The asset at path:
'Assets/Tools/Editor/AssetLock/GetAssetGUID.cs' is now unlocked.
You get this notification because you subscribed to this event. Make sure to lock the asset first.
The project in question is Project.

Cheers,
Your DVCS Admin Bot.

The last lock-related option is Break Lock and it is available only to administrators. Since users can edit the asset anyway, it has only administrative value.
The additional options can help with asset identification and communication. They copy the full asset path and GUID to clipboard.

Limitations Of DAL:

As you may have seen, DAL is not a panacea. It does not prevent actual asset changes, but that is only relevant if users chose not to obey the procedure. Breaches of changes are handled by Administrator on case-by-case basis at the time of integration of a pull request.

It still requires some communication when it comes to editing scenes and prefabs in parallel, and users have to be aware they are working on the same things. In some circumstances, it may be beneficial to lock even these types of assets, which is allowed.

Currently only individual assets are locked/queried, folders and multiple assets are not supported. The web server with PHP and MySQL is required.

Currently only Mercurial and Windows users are supported. If there is enough interest, it is possible to expand the functionality to OSX and git users.