For context, since a lot of people on HN haven't worked on games - this is not intended to compete with Git for general software development. This is a competitor with Perforce for game development.
Git is fine for text based files like code, but it's really bad at stuff like textures, 3D models, audio files, and other non-text files that game developers need to collaborate on. For example, one artist might need to obtain an exclusive lock on some art assets while editing them, because there is no sane way to merge two artists' async edits.
The SOTA in this area is Perforce (https://www.perforce.com/products/helix-core), a proprietary system. From what my gamedev friends tell me, when Perforce works it's great, but it hits enough snags that you need a tools engineer to manage it and occasionally fix issues manually. Git LFS is an alternative, but my gamedev friends all prefer Perforce especially when working on team projects beyond like 3-4 people.
Something else that git isn't good at: permissions. In gamedev, you might have proprietary work that you want to restrict to certain users. In P4, you can add restrictions to certain directories for only those who have signed the required NDAs. That's not something that you can do in git: it's all or nothing. Maybe you can set something up with submodules, but that's going to upend your repository if you hadn't planned for it.
I once worked in a git repository that required those kinds of restrictions.
This was within a bank and the code in question was related to enabling Apple Pay from within the banking application. The consequences of that information and code leaking or being seen by anyone who had not signed the NDA were very serious (don't remember the details but it made the lawyers were extremely stressed about it).
Needing to figure out a way to protect those parts of the codebase it was decided in the end that the "easiest" way of doing this was to split the repository in half, with the actual artifact building taking place from the half that had the NDA code. The rest of the application (basically the whole application) was then used as a dependency by it.
Still didn't quite solve the issue, but access to that repository was heavily controlled.
Strikes me as bizarre that payment code would be sensitive, unless it's a security by obscurity thing (which would also be concerning).
Keys, secrets, etc. yes. But code? What am I missing here?
As others have said, it's Apple and they do not take kindly to other people leaking their technology/announcements ahead of time.
See also: the time that ATI's CEO told his employees that their chips would be powering Apple's to-be-announced hardware a few days before the announcement. Steve Jobs responded by pulling all of ATI's hardware from its demo units at the announcement, not mentioning ATI at all, cancelling a joint demonstration of the Radeon card that was going to be in the system, and never partnering with ATI again.
https://web.archive.org/web/20001216031800/https://www.zdnet...
From the linked article, it was a press release, not just to his employees.
> The incident began Monday when ATI, which supplies graphics cards for all Apple's current models, issued a four-paragraph news release that stated its Radeon processor would be featured in three new Mac models -- none of which were announced by Apple (Nasdaq: AAPL) until CEO Steve Jobs' Wednesday morning keynote address.
Oh, I misremembered then. Yikes.
> and never partnering with ATI again.
Except of course shipping ATI hardware for years afterwards, then also using nvidia, then dropping nvidia and only using ATI/AMD until transitioning to Apple Silicon.
Well:
1. They kept existing designs, since even Jobs wasn't so crazy as to demand a complete re-architecture of existing laptop models on a whim; plus they probably also had contractual obligations/pre-purchase arrangements
2. They switched to nvidia, but from everything I know they also hated working with nvidia (IIRC Jobs accused nvidia of stealing Pixar tech)
3. AMD is a different company than ATI (technically), and Apple of that era was different than the Steve Jobs temper tantrum era.
But yes, relevant details.
Also Steve Jobs Apple is probably much different than today's Apple.
Sounds like a bit of a dick...
For violating an embargo and publishing a press release announcing products of another company that hadn’t been debuted? What “non-dick” response do you think is appropriate against a prospective partner that violated clear guidelines that defined their partnership which basically included “#1: Keep your mouth shut”, exactly?
They unilaterally issued a press release about Apple's upcoming release.
That's kinda a no-no for partnerships.
He was, but this incident wasn't an example. That's a righteous punishment for an infraction like that.
Going scorched earth was kind of Steve’s thing.
One word: "Courage"
Not sure I understand? Sounds like a temper tantrum to me.
Inside joke. Apple's marketing guy used the word "courage" as an explanation when removing the headphone jack.
lol
That decision had a lot of detractors at the time, but it ultimately seems to have been correct.
I don't know if I agree.
People who moved to bluetooth got arguably worse sound quality.
For wired headphones, there were very few lightning headphones and they mostly sucked.
Now with usb-c you can get a broader range of headphones (because other phones have gone with usb-c).
either kind needs a dac inside the dongle, or the shell of the headphone, or have a bulky external dac.
3.5mm is still king for decent wired headphones/earbuds.
Most people don't like wire and don't care about genuinely good sound through wire vs. artificially sweetened (DSP-ed) sound through Bluetooth. That's what Apple is targeting at, the mass consumer.
Wired headphones are making a comeback though. They're even considered trendy/fashionable now.
https://www.cnn.com/world/wired-headphones-comeback-spc
https://mashable.com/article/wired-headphones-instagram-acco...
Honestly that "most people" seems to be more marketing driven than real opinion.
When you simply can't purchase one thing, then you move to the next thing.
I have several wired and wireless headphones, all for different uses, and the existence of wireless headphones don't make the wired ones less useful for their particular strenghts.
It was wrong then, it is still wrong now.
All my three phones have a headphone jack.
I don't have or plan to have an iPhone.
It still has a lot of detractors but it is hard to beat such a sheer market flex.
Four words: You're holding it wrong.
I believe that the action does reflect Jobs' ego in the following way.
Namely, his belief that CEO == company.
Jobs would never take the view that the action of the CEO of ATI is actually one bad actor acting alone which doesn't represent what ATI wants as an organization, and is unfair and damaging to that organization and all of its employees.
The reason he would not take that view is because then he would not be able to believe that he is the single most important thing at Apple, overshadowing everything else.
If the leak had been the responsibility of some rank and file employee at ATI, with appropriate action taken against that employee by the ATI CEO, it is likely that Jobs would likely have reacted differently, because it then would not longer be seen as a personal matter between him and the CEO, where the corporations are just pawns in a game of teach-you-a-lesson.
Ultimately, it is the CEO who has to take responsibility. They are the chief executive officer, after all. And they are well compensated for it.
But did you notice how this was backwards: the company taking responsibility for the CEO's actions, in the court presided by Jobs.
It’s funny how exciting Apple Pay was when introduced, only Apple pulled the lock-everyone-in card and now we’re all using QRcodes.
In the US you can use Apple Pay anywhere NFC payments are accepted. It's generally completely open on the acceptance side at this point
It's anywhere close to open and requires vendors to seek approval from Apple for every implementation - Apple just has the market share to make everyone dance to their tune.
Apple Pay works on any terminal that supports NFC tap to pay payments. There are thousands, tens of thousands of terminals in the US that are not "Apple Pay" that you can use it on
Same here in australia. We had NFC payments ("tap and pay") since ~2006. Nearly a decade before apple pay launched in Nov 2015. For us, apple pay was like "oh, I can use my phone instead of my credit card? Neat."
People use it everywhere here, just because its easier than carrying around a card.
Most developed countries I've been to use NFC/Apple Pay. Where you see the use of QR codes is in China, Africa, and other countries that VISA/MC/AMEX hasn't penetrated (for either political or socioeconomic reasons).
I'm in HK, most HKers use ApplePay with NFC, and don't use QR codes. Over in mainland China, however, it's all QR codes (with AliPay, WePay).
I guess the "we" depends strongly on location - I've no problem using Apple Pay basically everywhere except Walmart.
No idea what this is referring to...
Many countries in the world have local systems for quick, direct bank transfers encoded with QR codes to be used by local payment systems or local banking apps.
SHEESH
Because it's Apple. They are huge, have scary lawyers, write scary contracts, and want to "delight the user" with features only when they announce them. They hate leaks, and demand separate teams for basically any/all development.
> security by obscurity thing... What am I missing here?
You are looking at the problem from the wrong direction.
If you build a honeypot, to trap hackers, does it behove you to explain what the bait is, and how the trap works?
Know your customer, fraud detection heuristics, finger prints, behavioral triggers are all areas where banks, and financial institutions need to keep the sauce secret. Telling the other party "how" you catch them just gives them the steps of what not to do.
Alternatively, you can also just try not to have security issues, and resolve them with urgency when they do pop up.
The code revealed the existence of Apple Pay, which had not been publicly confirmed.
It seems this wasn’t about the code itself, it was about Apple Pay not being announced yet. So only people under NDA would be allowed to even know what they are working on.
It's kinda like that, there could be a proprietary fraud detection heuristic in there that you don't want to get out.
The non-strawman version of "security through obscurity" is the belief that a system is secured by means of keeping its mechanisms secret.
Suppose an organization doesn't believe such a thing; it's still more secure to keep code secret than not.
Obscurity is a valid layer of security, just not a valid corner stone or linchpin of security.
In particular, when code operates as a service (end users don't have the executable code on their machines) then protecting the source code is a real security measure. Without it, attackers can only probe the service as a black box, guessing about what it is doing.
Yes Apple Pay relies on security through obscurity even today. See the Veritasium recently made a video about it https://youtu.be/PPJ6NJkmDAo?is=iUuJ0W9xUHF_6gTU
There is a bunch of mundane stuff in the banking/finance world masked off by paperwork.
It's not sensitive in the leaking state secrets sense it's sensitive in the risk adverse lawyers on both sides think it's sensitive.
The Bureaucracy exists to perpetuate the Bureaucracy.
Maybe that’s some scoring to decide if you should be able to pay or not with some method.
Can confirm split repos is an excellent solution for protecting IP.
PCI DSS has various controls for code handling credit card cards which tends to require different workflows for code that touches credit card numbers, from say, marketing pages. So splitting the code into different repos can be quite common.
Not sure what it is on the Apple Pay side but with FPLS it is/was basically your keys would be revoked and you would be ineligible to ever get new ones… so no content that requires DRM on iOS for the life of the company.
Banks absolutely love security by obscurity. No clue why.
It had nothing to do with security - it had to do with contractual obligations. Contracts with Apple (also Google, Samsung, Mastercard and Visa) required the product to be kept absolutely secret before the public launch. I was a tech lead with developers working on Bank of America’s ATM client - which had firmware and software updates ahead of launch - and I found out about Apple Pay the day it launched. Across the aisle were developers who supported the debit auth platform and they had no idea either.
security by only obscurity is bad. Having both is better.
For example say I have a hollowed out wall that is hidden behind a painting.
Just putting my money in the hole is bad once it’s found it’s gone but if I put my money in a safe in the hole. Well now you need to find it and break the safe and a hidden safe is objectively better than just having a safe on the floor because you need to find it first.
Sure, if there's many paintings scattered around the house of various sizes, but if there's only one painting, in your office, behind the desk, mounted to cover a safe at standing height, then you might as well hang a neon sign saying "Look Here!" next to it.
Banks run a lot of code as a service. Most people never get to see the executable code. If the source code for those services were easily visible to employees that don't need to have access to it or external consultants and such, that would be monumentally stupid.
It's the most effective kind of security
I assume compiling the payment code as a library (w/o debug symbols) and making that available to the wider group wast allowed? Header files a problem?
Could you not have placed the code in a submodule and likited access to the submodule repo ?
That's by design of git, you can't forget that git is first developed for a bazaar model of information flow, especially with a big decentrailized project like the Linux Kernel, not the silo and isolated corporate NDA and closed model you described. Git prefers open information and discourages information closures and segregation of information by placing restrictions exactly like this.
Git enthusiast would often tell you to do this separately with a submodule, and set permission on the version control forge software level (which means Gitea/Github private RBAC access to certain repos for cloning), sure, but that is also painful as hell.
But my point is that all of this is exactly by design from Linus Torvalds's need for Linux Kernel to replace BitKeeper. Git simply isn't the tool for everything, it was developed for a software project with liberalism in mind, but corporate stuff is monoculture and prefers proprietary, shut-in model, and the eat your own dog food mindset, and no wonder it is so painful to deal with.
Git submodules aren’t convenient either. For the silo and corporate development use case, just use multiple repositories and make your build tool aware of multiple repositories. It is slightly less painful than submodules.
I feel like submodules could be a lot easier to work with if the git command made it easy to update all submodules in one go based on branch head for the submodule.
That and a way to recursively commit/push (i.e. make a change within a submodule, commit in one step with the same message in the parent submodule).
Right now, if you want to push a change to a file in a submodule such that it propagates to the users of your repo, you have to:
1. Change the file
2. Commit within the submodule
3. Push the submodule
4. add the submodule change in the outer repo
5. commit in the outer repo
6. push the outer repo
In the Beagle SCM, I am doing my best to simplify that flow, but that is only doable because Beagle has multi-project repos. Still, there are some difficulties with submodule recursion. In the git model, difficult to imagine this working smoothly.
You can combine the push step. A submodule intentionally follows a different development/commit/version cycle, otherwise you are supposed to use a subtree.
git submodule update —recursive —remote is that, isn’t? Perhaps throws also an —init if it’s the first time.
To add to this, you can add in your .gitmodules the name of the branch that the --remote flag will follow. It just works.
[dead]
You can set submodule.recurse to true and then git commands operate as if you passed --recurse to them? It's just that most people don't actually want that, because a submodule is generally something you intentionally want to handle separate.
Submodules ARE multiple repositories. This is not an either or.
I understand that. I am just arguing that making your build tool aware of multiple repos is enough; no need to use submodules in git and make your build tool pretend to operate on a single repo.
I think everyone knows that this is a consequence of git's design. Nobody's disputing that.
Unfortunately there are many people who think git is a panacea and is suitable for all version control tasks of anything.
And a few of them think that it is corporate structure and development models that are actually the problem.
I'm just asking questions!
I don't understand what you're trying to imply.
If you're making the point that there are multiple confounding factors to just about any non-trivial problem, then I agree.
No, I'm making the point that the way in which git doesn't fit well into many corporate processes can be interpreted as more of a negative commentary on those processes than on git.
I ended up writing my own layer over git for permissions for a specific client a long time ago. It has a huge amount of useful features - sadly, I never took the idea further.
Turn it into a business
That always seemed crazy to me about git. Permissions are a pretty basic enterprise offering.
Does Gitlab do better with this?
I guess it's because git wasn't developed as enterprise software.
This. People forget a lot of Git's design philosophy harks back to the ethos of open source development. Enterprise features have made it in over the years but still mostly with the FOSS development workflow/model in mind. Also why the most enterprise-y of features (like LFS) are add-ons rather than core.
It's pretty sad in the indie gamedev world that handling and versioning binary data is considered an enterprise feature. I understand that git was explicitly designed for source code, but it would be nice to have any open system that handled versioning binary files well. This is pretty much the only reason I am going to try out lore at some point, although I am not super psyched at some of its implementation.
Maybe you could go to linus and demand enterprise features and support.
;)
How is it crazy? It's perhaps not granular (the repository is the boundary, and that's that), but you can definitely restrict who can pull or push as easy as you can make rules for SSH.
Plenty of not-very-granular "enterprise" systems out there, it's not exactly unique to not always have full ACLs on the smallest of objects.
git repos viewed through gitlab's slow Ruby monolith are still git repos, so it doesn't make any difference.
No.
The maximum granularity is also 'per repo'.
Because code is not supposed to contain parts that are secret or specific rules: those are data, that your program should work on. Git is coming from the open source movement.
> Does Gitlab do better with this?
Not exactly but if you're not obsessed with maintaining a monorepo, Gitlab allows you to organize your repos around organizations, which then has granular permissions. The underlying primitives is still Git, of course, so you can just submodule as necessary.
> That's not something that you can do in git: it's all or nothing.
That is partially incorrect; you can restrict writes via hooks but not reads; you'd need a workaround like submodules
Does `--no-verify` override the restriction via hooks, or are there some kind of server-side hooks that can be used?
no-verify cancels local hooks, remote hooks are unaffected.
Gitolite supports per-diectory/file write access natively, for gitlab you'd probably need to write your own.
Interesting. I'll have to do so reading into remote hooks!
> Something else that git isn't good at: permissions.
It doesn't have to be good at permissions. That's what DevOps platforms that integrate git are for.
Your comment and the parent comment are fantastic. I'd never have thought of them because I've never worked in the field
The way I usually solve this is by using git submodules.
Oh man, I've been laughing at this for 37 minutes straight now.
Git submodules are the regular expressions of version control.
When you use regular expressions, you now have two problems.
When you use git submodules, you now have five or six problems.
And every PR turns into a triple recursive reverse merge.
AFAIK the issue with using submodules is you still need the rights to pull the other source repo. However, you can use submodules or LFS to pull a specific build artifact from a build artifact repo or source instead of the source repo, which provides a neat way to manage the dep without fattening the main repo and allows the source repo to be kept separate and high security. I'd certainly do this before changing RCS/VCS solutions. That said, reverse engineering has become relatively trivial in the AI age so the practical utility of providing built rather than source elements is dropping.
You'd have the non-NDA, core stuff in the submodule. The submodule can then be referenced by the big, driving NDA repo. And maybe another repo with a non-NDA, likely simpler "shell" that lets non-NDA devs work with the codebase.
If you need to NDA the core stuff instead and thus can't pull it as a submodule, the only thing I can think of is to pull the core as binary/compiled artifacts.
You can set up submodules to not pull the other source repo by default tho
Try gitolite?
https://gitolite.com/gitolite/index.html
It has fine-grained permissions but works with regular git clients.
Does it actually restrict read access to specific directories? Unless I missed it, I didn't see anything in the docs about that.
"write access controlled at the branch/tag/file/directory level, including who can rewind, create, and delete branches/tags."[1]
[1] https://gitolite.com/gitolite/overview.html#what-is-gitolite
I said read access. The point is that there’s restricted information that only some can access.
My teaspoons are terrible at peeling potatoes.
Git has no built in authentication or RBAC. Thats not what its for. Its flat file source control.
I swear loads of people havent a clue how git works or why it exists...most of the git based cloud services out there are 90% additional crap bolted on.
I think that’s the point of the OP. Git is great at certain things but is not a one size fits all.
>Thats not what its for.
This is a weak argument you could use for any missing feature.
But it's more like a fundamental design choice.
Obviously, and that's why tools like perforce and lore exist. What's your point?
Perforce predates Git by a decade (but your overall point is valid)
[dead]
Doesn't git crypt solve this? You can have encrypted blobs in a repo that will be auto decrypted if you have a working key.
That depends on you distributing working keys for any components you want to restrict access to, and managing those keys for all users, revoking them when access permissions change, etc. It's a lot more complex, more work, and harder to manage than centralized RBAC or similar.
People don't use git crypt nearly enough unfortunately.
Agreed. I use and love git crypt, but it doesn't get enough use. I think because it's easy to screw up gpg keys. Most of my uses (for one to three devs) have become symmetric keys shared out-of-band instead of using gpg keys because we've had lots of onboarding pain even from people who are quite competent. There are just a lot of sharp edges in gpg that you don't know when you don't know.
i really want to use a similar tool that uses age instead of gpg, but the ones i've tried were all not as nice to use, or very undermaintained. but idk i havent checked in a while
Not really, precisely because it’s decentralized. You can’t audit whether a user accessed one of the hidden files, or really even who can access it once you accept the reality of the risk that some team will put a key on S3 or a shared drive or whatever.
It’s fine for things that you want devs to be able to see without the Git host being able to see them, it’s less good at RBAC because there’s no real “identity” component at read-time.
You can use Mozilla SOPS instead with IAM roles and KMS instead of gpg. They also shifted to AGE over gpg.
What a soup of acronyms
It’s the same problem. You don’t have an audit trail. That’s needed in a lot of situations for compliance reasons
Why is an audit trail interesting? I would expect its basically useless as anything that can be accessed can be copied. So it doesn't even give info of user x viewed y at some time.
Git submodules + SSH keys is another (somewhat "homebrew") solution to this.
Confirmed. Perforce is (was?) the industry standard for most larger game teams. If you've been in the business for any length of time you're probably pretty familiar with it and all of its many warts. But Perforce does get the job done and it's reliable and stable.
However combining some of the flexibility and workflows of git with the ability to deal more efficiently and effectively with large asset files is something that a hell of a lot game engineers would be interested in. And having the virtual checkout as a feature out of the box for folks used to half terabyte repo sizes is definitely a huge plus.
This announcement is definitely a big deal, and if the promises for lore actually measure up, we could be seeing the beginning of a switch over to open source version control for larger asset heavy games where git was still not a great fit.
Will be interesting to see what the public code hosting platforms do (github), and whether there will be any major structural changes to git (I'm thinking not likely). I wonder if Epic will make a play to take business away from github or gitlabs.
There is work being done on Git to add pluggable object backends[1] by some GitLab devs that could changes things up a bit and make large object handling not suck Maybe Lore could act like a promisor remote to pull large objects from as well for interop.
1: https://gitlab.com/groups/gitlab-org/-/work_items/15061
> Perforce does get the job done and it's reliable and stable.
Well… perforce does get the job done, but it does need a lot of hand holding from an admin (and you need someone who knows p4 server if you’re using it). It’s also stable in the sense of “it has exactly the same feature set as it did 15 years ago, and branches (streams) are considered modern”.
Its important to understand that in Game Dev a ’git clone’, aka ’p4 sync’, can be a terabyte of stuff.
Git is bad at such volumes of binary assets, textures, models, sounds, etc.
Yikes, does that mean each dev has terabytes of stuff on their machine?
If they're doing a full build then yes, but in practice most devs or artists will work on smaller partial checkouts and another engineer or team works to keep everything integrated into regular builds. You don't need the full game locally to rig a model or paint a texture, and even some gameplay stuff can be tested in a graybox demo environment without the full game assets before being integrated.
Theoretically yes, but in practice you don't need the full repository, but even in mid-2010 (when i first encounter Perforce in a gamedev company - previously the companies i worked on used Subversion which was also able to handle some huge repositories -- one had all their games from the 90s to early 2010s in a single Svn repository, including code and data) a workspace with just the game's code and data in the engine's format (i.e. excluding everything a programmer wouldn't need, like the "source data" for assets such as max/maya/etc files) was around 250GB or so. IIRC the entire repository (all versions, not just latest) had crossed a petabyte in size :-P though that company put absolutely everything in Perforce.
One thing I don't like about Git LFS is that there is no way to delete very old history. It's the 'git spirit' to not allow deleting history, but in the context of LFS it sounds horrible. Especially if you use Github.
If there is an asset that is updated very frequently in the early stage of development, you'll be charged for all the storage for the rest of the repo's life. That happens a lot in gamedev: most assets go back and forth early on but once it's done no one will touch them ever.
> One thing I don't like about Git LFS is that there is no way to delete very old history. It's the 'git spirit' to not allow deleting history, but in the context of LFS it sounds horrible. Especially if you use Github.
At my dayjob we used Git LFS for a bit, but foud it unworkably clunky - we eventually found it easier to just make a separate "LFS" repository and add it as a submodule to the main monorepo. Now we can rewrite the history of the LFS repo on an as-needed basis.
> Now we can rewrite the history of the LFS repo on an as-needed basis.
With the giant caveat that doing this effectively breaks the history of the parent project. TBF that's not really any different than rewriting history and later discovering that an old version of a lockfile no longer works but I still think it's worth mentioning.
Yeah, I can't edit my first post, but should note that if you don't want to rewrite history of the submodule, it also at least lets you consistently do shallow clones of the submodule - various git-related CI/CD operations on the main repo won't work with shallow clones, so it's a pain if you have to pull in a million old versions of binary dependencies just to run gitversion.
As far as rewriting the history of the submodule goes... that's really something we're okay with - for us, the alternative to the git submobdule was just an SMB share - indefinitely keeping old binary dependencies isn't something we want to do.
> that there is no way to delete very old history
That's one of the features of Git LFS is separately managed storage and lifetime.
You are correct that GitHub does not offer that feature.
I think you can delete the history but it’s hard (more than three steps), at least that was my conclusion from the e blogpost below
https://janetgilbert.net/staging/2026/04/27/dealing-with-old...
Git LFS is unusually bad. I think the maintainers are trying their best but it is one of those things where there are so many bugs and so many users that they will claim they are overwhelmed with LLM authored PRs instead of conceding that they own a very valuable piece of real estate (being installed alongside git by default) and totally mismanaging it.
LFS is not installed by default alongside git, it's a separate package in most systems
P4 is more "industry standard" than "state of the art"... But it does handle large files and partial checkouts without feeling bolted on.
Git has had native partial checkout for ages.
Do you know any large, non-technical teams that use it? The workflow is just not feasible even if the feature technically exist.
Things like submodules become even more complex. Without submodules there's no equivalent to map in library/engine code into place in a way where you can push changes back upstream.
Last I checked, things like logs and pulling still operate on every commit so it doesn't really scale the same way.
> partial checkouts without feeling bolted on.
OP said this, maybe Perforce's partial checkouts are just better than Git's.
I am building a small asset heavy game. Ran into a similar problem. Built a storage cost efficient tool for exactly this [1].
[1] https://github.com/debarshibasak/assets
Man, I always hoped to find a project like this on hackernews. I starred it.
Git LFS is a major PITA, and if you use GitHub is even worse since there are quotas and rate limits that are charged separately.
One of those ideas that sounds clever in theory but in reality doesn’t work very well
Also... it's kind of weird taking a decentralized system and recentralizing it.
Isn't centralizing systems over time more common than not? Another example is the Internet: websites do not have a uniform distribution of traffic
P4 is also really well integrated into IDEs and UE Editor so that I don't need to think about it as much as I need, compared to Git. Locking assets, releasing them, merging into streams etc., is overall pretty streamlined. When it works, it's great, but when it doesn't work though, it's pretty hard to diagnose issues.
One of the many points of git's design is that most of the steps you need to do in P4 are not necessary at all. You do not "lock assets", you do not "release them", there is no "merging into streams" equivalent.
The entire workflow with git avoids huge amounts of the cognitive load of using P4, which in turn means that integration with IDEs becomes much less important.
I worked with P4 around the time I first started learn git (coming to both from SVN). P4 struck me as "what a for-profit corporation would imagine a VCS should be like, if they'd never seen git". So glad to be far, far from that particular tool now.
Solving merge conflicts on text-based files is infinitely easier than binary-based. It's not a useful comparison.
What does that have to with your VCS being integrated with your IDE? Isn't resolving merge conflicts in binary files going to entirely a function of the IDE and not the VCS ? What am I missing?
Merge conflicts on binary files are typically unresolvable in practice because the editing tools don't normally have the ability to compare files. This leads to one of the two divergent sets of changes getting lost. That's why great VCSes allow file locking: locks are a communication tool between team members to avoid losing work.
a great VCS wouldn't require locking or pre-communication; it would simply refuse to even try to merge assets it couldn't deal with, and leave the result of a merge in a state where the user can reasonably address it (in this case, presumably just "yes, use mine" or "no, use the one already in the repo" and perhaps "use mine but rename it").
I don't get it. Now your options are to lose your work, or for somebody else to lose theirs, or, maybe an un-fun manual merge process - and this after you've both done the work! With a per-file lock, on the other hand, you can't start work until you've got the lock, which prevents others from starting work on that file in the meantime.
(It's usually possible to bypass the locking mechanism, but this is not a habit people get into, because the end result is the situation you describe, and nobody wants to end up there.)
We seem to be in some kind of local minimum here, but perhaps LLMs might help us escape it by facilitating the creation of more merge tools.
> You do not "lock assets",
Uhhhhhh. Locking binary assets is ABSOLUTELY NECESSARY in Git. Except Git can't actually do that. So what locking binary assets Git looks like in practice is an unenforced message in Slack saying "hey I'm changing this file please no one else touch it".
Git's design provides zero value and zero affordances for solving the very very real problem of unmergable binary assets.
> there is no "merging into streams" equivalent.
Weird take. Streams is Perforce's mediocre take on Git branches. P4 stream merge is close enough to Git branch merge that I would say they're in the same family.
> P4 struck me as "what a for-profit corporation would imagine a VCS should be like, if they'd never seen git".
And yet Git has almost zero mind share in gamedev because it doesn't solve the problem.
P4 is somewhat mediocre. And it's made zero improvement in ~8 years since getting bought up by PE.
But I can also teach a game designer or artist who has never heard of source control how to correctly and safely use P4 in about 15 minutes. Meanwhile you can tell Git is badly designed because there are 10,000+ tutorials explaining how easy it is! Spoiler: things that are actually easy do not need 10,000 tutorials telling you that it is easy.
> Git's design provides zero value and zero affordances for solving the very very real problem of unmergable binary assets.
OK, I think this is well-established by up-thread comments, and I don't disagree with it at all (though note that its not just binary assets, it's anything that isn't line oriented, which includes for example XML).
I didn't realize that LinearIO's comment was really that specific; it appeared to be describing general properties of P4 unrelated to the binary issue.
The 10k tutorials on git might indicate issues with its design, or it might indicate its massive popularity. Hard to say.
I just remember that learning to use P4 required learning a ton of concepts for what P4 thinks your workflow ought to be; learning git has largely just required a simple 1:1 mapping between git commands and the things I do with VCS 98% of the time.
> The 10k tutorials on git might indicate issues with its design, or it might indicate its massive popularity. Hard to say.
It's "hard to say" if you want to ignore the fact that it requires 10k tutorials. Meanwhile, there are 500+ person companies with non technical users using P4 with literally 0 onboarding other than "by the way, undo is broken don't use it".
> I just remember that learning to use P4 required learning a ton of concepts for what P4 thinks your workflow ought to be; learning git has largely just required a simple 1:1 mapping between git commands and the things I do with VCS 98% of the time.
Using P4 is: Download P4V, install plugin for $EDITOR, and double click on a changelist to submit.
the fact that 10k tutorials exist says way more about the human urge to write, document, and share information than git itself. the 10k number is also a made up statistic. It might be 2k, it might be 20k. i use git daily and often fairly intensively, and i've read perhaps 5-10 tutorials, so the idea that "git requires 10k tutorials" just doesn't make any sense.
if anything, the fact that something with good search-fu might find a tutorial more specifically catering to their background and needs seems like a huge plus for git.
> Using P4 is: Download P4V, install plugin for $EDITOR, and double click on a changelist to submit.
"a changelist" ... "what the hell is that? I've been editing my files, I don't have "a changelist"."
and anyway, joining a company that's already using P4 is never, ever going to be that simple (even if it was actually that simple as the first user, which I question)
Honestly, when our backend team merged into one that was using Perforce for the backend learning how to use Perforce wasn't realistically even a blip on the radar of what to get used to. I was against it at the time for what we were doing but with the benefit of hindsight I can say that I prefer something like Perforce if someone can manage it for me, or it's a set-and-forget type situation; I don't personally have a lot of use for the distributed part of DVCS.
Currently I use Fossil for most projects, but it's not a compelling choice (just like git) for when you have binary stuff. You've got `fossil uv` for unversioned files, but I think I would rather just sidestep the entire problem with a better versioning model than what we've settled on for text files.
> Honestly, when our backend team merged into one that was using Perforce for the backend learning how to use Perforce wasn't realistically even a blip on the radar of what to get used to.
I think a lot of what people are assuming here is that because git has such a steep learning curve, P4 does too. You can literally ignore all of the concepts that P4 has and still muddle your way into getting things updated.
All I can say is that wasn't my experience with P4 in 2015.
> But I can also teach a game designer or artist who has never heard of source control how to correctly and safely use P4 in about 15 minutes. Meanwhile you can tell Git is badly designed because there are 10,000+ tutorials explaining how easy it is! Spoiler: things that are actually easy do not need 10,000 tutorials telling you that it is easy.
100% and LOL. Sometimes I wonder where we'd be if Mercurial (hg) had won that contest.
Hoverjets, probably.
If you're not sick of hearing about it, jj is git-compatible, and borrows heavily from hg's UI and terminology. It's CLI is concistent, logical, and easier to pick up than git.
There's also the virtual streams (branches). A mapped overlay over the actual contents of the repo, which you can check out and get a subset of the contents. Or even can provide different files that have been mapped in the server for each particular virtual branch.
This is used so e.g. an artist gets a repo that contains sources for the art assets, while a programmer gets the same repo but instead of art sources, it downloads the already produced binaries. As a SE, you just want to build the code, and don't care about 800 GB of art asset sources.
Another thing git isn't good at is massive codebases with long histories. IMO git should have a configuration option to pull commits lazily without the need for `--depth` and the `git fetch` dance that goes along with it.
There's `--filter=blob:none` and it allows to automatically fetch blobs when needed.
I wish it was as simple as `git clone <url> --lazy`
I had previously worked on a big tech monorepo that has gigabytes of history. It would take forever to clone or do operations on. I had a cheat sheet of git commands that would do things lazily but I forget them (which is the issue).
There's also a new player called diversion (diversion.dev) which I think may be a YC startup? Anyway it takes a different approach of being more like Google drive but bringing in VCS behavior making it more indie and designer friendly.
At my previous game-dev-company job we ended up splitting things up into:
1. Code - Git
2. WIP art, shared assets (logos, marketing materials, etc) - Google Drive (because things are often changing, getting passed around, etc)
3. Finished assets (PSD files you're done with, or you think you're done with) - SVN (because we wanted a log of who contributed to what, wanted artists to be able to pick up where someone else left off; having a log of who made changes to a given PSD)
4. Assets rendered out to PNG to include in the app bundle/publish to the static file servers - Git (because those files never changed after being published so the git history wasn't polluted with unneeded files)
I've also used LFS, which is... a fine workaround, but still not great. Users who don't have it configured can still commit binary blobs; users who don't have it configured will clone files incorrectly; if the LFS server is slow, unavailable, unreliable, then the system starts to behave oddly; you need a Git server that supports it.
It was a huge hassle to manage; having a system like this would have been a godsend at that company, and if I still worked there I would be spending all day importing our codebase and assets into it to see how well it works.
Yeah that seems like an awful solution. This is exactly why we use perforce and just shove _everything_ in it.
what happens if WIP stuff has rapid shifts or changes? art direction changes on the product level, etc? or even something as simple as an asset designer quits or get sick for a while
SVN makes sense cuz it's done and dusted, but I could see the Drive gettin real messy real fast if things change a lot
Why was SVN preferable over git for finished assets?
A significant part of my job, unfortunately, is helping people fix their workspaces when Perforce (p4) goes bad, or creating guardrails and wrappers to stop Perforce doing bad things.
In fairness, p4 predates most of the VCSes we consider "modern", so I empathize with a lot of the underlying architecture decisions. However, it has and continues to utterly fail at improving at a reasonable pace.
For example:
> Scripting p4 is a nightmare
This is why I wish more command line tools were split into a library that does most of the work and a cli module for purely user interaction. Parsing stdout seems so unnecessary and could be avoided if a program could simply import a library.
There are various language specific wrappers for the API - the Python one is good, and now there is a Go one too.
> p4 predates most of the VCSes we consider "modern"
p4 also significantly predates VCSes we consider obsolete. p4 is almost a decade older than SVN.
20yrs ago, for me migrating off p4 onto svn was such a relief and feeling really "freeing" in a way I haven't felt often.
A quick reply;
> p4 tracks file metadata of client workspaces on the server
Honestly, using P4 necessitates a plugin for $EDITOR of choice. The jetbrains IDEs do full on integration, but I managed a decade with an addon that just ran "p4 add/edit" on any save.
> Scripting p4 is a nightmare
Agreed, but so is git, or plastic. My experience has been using the CLI with -Ztag is the way.
> By default, p4 "helps" you with text files by "correcting" line endings on sync or even converting between encodings
This is a nightmare, and definitely one of P4's worst traits. Everywhere I've worked we have a presubmit trigger, to catch this.
> By default, p4 keeps flies read-only, only unlocking them when explicitly marked as being edited.
See point 1, but honestly you go very very far with just p4 edit a whole folder.
> Branching a modest game project, with, say, Unreal source code, can take hours. And this is the quick version where you ask the server to simply create new metadata, with no file transfer to a client.
A new stream including submission at my current job is about 20 minutes. Even at $BIG_CORP with 800 people working on a project, a branch didn't take hours. (Assumning by branch you mean streams. If it's a branch I have no idea, we don't use them)
> p4 is licensed by the user-account
P4's licensing is predatory. It's very much "talk to us", but you'll find out that "talk to us" pricing is exactly the same for absolutely everyone. If you're really lucky, and get someone who is feeling generous, they might give you 1-2 extra automation seats. Given P4's security model is so old, I don't hate shared credentials for this. A bit like IAM roles, nobody should really have access to the credentials even if they were scoped appropriately.
I've been using p4 daily for about 21 years at various game studios (with a small break of around 5 years in the middle, where I used it only sporadically) and I think I can count the number of times I've used "reconcile offline work" on one hand. I think the only time my workspace has gotten into some kind of corrupted state was because of a crash that left it half-synced. If that ever happens I usually just blow it away and force sync the entire repo again rather than use reconcile (because it's so slow).
I’ll add some more
- The P4 cpp api was apparently designed before any modern Cpp std lib was available. And is at best archaic, and stringly to use.
- P4 encoding support is pain in the ass to configure. And ensist on adding or removing bom to files.
Modern cpp is a bit of an oxymoron
Reconciling is my least favourite part. It always feels like everyone’s checkout is a unique blend of local files and permissions you have to fix up now and then. It can be hard to keep track of how much one’s deviated.
I can understand how git isn’t always appropriate for less technical workflows and large file sizes, but p4 pain is its own character.
I've had the misfortune of scripting an UE build with perfoce as the VCS, it truly is abysmal.
> this is not intended to compete with Git for general software development. This is a competitor with Perforce for game development.
Well, it is intended to compete with Git for version control. It's just that Git happens to be so bad at some aspects of version control that it isn't used much in those cases.
There's no good reason that Git couldn't be good at versioning binary files, or splitting up large projects. I mean people have tried - there's LFS and submodules. It's just that those both suck balls.
I was kind of hoping Jujitsu or Pijul would take a stab at these major Git deficiencies but unfortunately it seems like they are content to do them as badly as Git does.
It feels like a Pareto Principle problem. 80% of source control is text files that can be three-way merged as text files, but a lot of hard problems are in that 20% that isn't.
Git does very well at the 80% and with tools like custom merge tools and git lfs/annex and git sparse "cone" checkouts can get pretty close to hitting the 90 or 95% case.
But yeah, so many of those extra tools in that 80 to 90% area are awful to work with because they aren't the default, aren't out the box, are hard to configure and get right. Partly because it always seems like there will be a gap in that 95%-100% window and partly because the use cases that need that 80% to 90% often are only "just 10% of use cases".
(Which is also to say that to survive Jujitsu and Pijul and others seem to have to work to make sure they handle the 80% base case extremely well just to compete with git, they haven't necessarily time to think about the 90% or 100% problem.)
(ETA: And also relates to why game development seems to feel the 20% cases more, because by volume of data game development is certainly closer to a flip of the 80/20 sides with 80% or more large binaries by volume.)
I would say that at least the submodule problem is more like the last 45%. Every single company I've worked in has ended up using submodules and it always ends up causing huge amounts of pain because they suck.
The only alternative Git offers, which is slightly better IMO, is monorepos. But they're only slightly better - they also have really significant downsides.
I'm 100% sure there's a much nicer solution to the kinds of problems people use submodules for that isn't submodules, but as far as I can tell zero people are trying to find it, despite it being such a universal problem.
From my viewpoint, a lot of the submodule problems are effectively same as the monorepo problems, including how you end up with both problems. There certainly are factorings of projects into "microservices" and moving the dependency management problems out of source control to package managers of different types. Microservices and package management sure do have their own problems and trade-offs and there isn't a "universal solution", just what side of the trade-offs you prefer to be on.
That it is difficult to merge e.g. media files has nothing to do with Git. It's just that it is one of the core assumptions in Git, that it is fine to diverge locally and cheaply, because you can just merge. I think this can't really be solved without a centralized server and that is just something, that Git doesn't want to be.
Yeah, decentralization is an important principle to git and tools to manage concurrency such as "file locking" become a lot harder to do in an a decentralized way than a centralized one. (Pijul has the same problems because decentralization is also an important principle to it.) It is something of a problem space that transcends beyond git. But I also think this is how we've arrived to this moment where the best source control tools have decentralization as an important principle. That 80% case with easy mergeable text files is a lot nicer to work with in a decentralized world because it is "offline-first" and quite capable. It often seems, across a lot of different styles of software development, worth the trade-offs that things like media files are harder to work with. (Again, realizing the obvious problems with game development where that often flips and the majority of work is often in the media file assets as much or more than the text files.)
I wonder how useful this could be as a generalized version control for regular user systems, as a way to rollback, or scrub through history. Presumably if this is designed to work at Epic and Big Game studio scale, it should work at home computer scale
This presumption has destroyed far, far more companies and projects than the opposite assumption (that something built for small will scale to big, then doesn't).
Can you name five?
Kubernetes and the one I'm working at right now.
It is truly a tragedy that the story for scaling Kubernetes beyond 3k-5k VMs is "run multiple clusters'. It's 10% overhead for tiny scale, doesn't scale well to large scale- it works well in this happy midrange of 7-3000 VMs.
(If anyone is about to ask why you would need that many VMs - many companies who do extremely large scale infrastructure! The kind of software where when it breaks it can create a crisis for utilities, governments, healthcare systems, militaries, etc.)
[flagged]
Git is certainly not great with binary assets, but calling perforce SOTA ... ouch.
If perforce is the best there is out there for large binary asset management, then there is a blue ocean worth of potential improvement for git.
Perforce is a piece of crap, a relic of the 20th century that must die in a fiery inferno.
It can be SOTA and still be garbage if there's nothing better (and there's nothing better, sadly). This is extremely exciting for anyone who's had to manage revision control for game devs.
I’ve spent more than a decade working in games and unfortunately perforce is the best out there for a variety of reasons. None of them are good.
15 years ago, both Google and Microsoft were on perforce. (The latter through a fork with a different name.)
Google still uses Piper, which started as a Perforce clone, though many people use it through a frontend like `fig` (try pronouncing ‘Piper HG’) or `jj`.
I like putting it like this: VCS can be centralized or decentralized, and it can version per file or per commit. For games you want centralized per file versioning, like Perforce. Git is decentralized per commit versioning.
I mostly agree. I think for games you want per commit versioning 95% of the time. You don’t _really want developers building with cherry picked old revisions etc, but you don’t want an artist to have to pull down the 8TB of new art content that was submitted in the last 48 hours to update a texture.. the only way to do that is per file versioning!
Indeed. But luckily those who mostly want per commit versioning are coders, and they are technical enough to find a solution for this. Like perforce git interface.
Ah, that's interesting, the requirements are similar in the CAD industry. Dassault Design Sync is used a fair bit for semiconductor design databases, for instance. An open alternative would be welcome!
Edit: I do feel a bit uneasy about epic games, though.
Can confirm, we have a team dedicated to the care and feeding of p4.
Git LFS causes nothing but pain, suffering, and regret.
I braced myself for this, and instead found that it worked surprisingly well for my uses. I only manage 4 gigs though.
> Git is fine for text based files like code, but it's really bad at stuff like textures, 3D models, audio files, and other non-text files
Git-annex ?
You gave me a flashback to that time I tried to use Git Annex on a moderately sized dataset and it resulted in a sysadmin at rsync.net personally contacting me to see if I needed help with whatever poorly written script was hammering their service.
In other words: No, absolutely not.
> Git is fine for text based files like code, but it's really bad at stuff like textures, 3D models, audio files, and other non-text files that game developers need to collaborate on.
How well does Mercurial work in this situation (or even Subversion, given that Perforce is non-distributed like SVN)?
Mercurial is a bit better, but not by much it had slightly better large file support historically but I wouldn't recommend it.
Has nothing to do with Perforce being the Oracle of VCS because it’s baked into the big 3? Riiiight.
Perforce is more the IBM of VCS. Older than it has any right to be. Has quiet "dark matter" support contracts with a lot of companies you wouldn't think need Perforce, but they've been using it for long enough they aren't going to change. Some of those support contracts included complicated forks and homegrown solutions that are so sunk cost as to be nearly black holes (and sometimes so different from baseline Perforce as to be evolutionarily different species).
Well another factor could be that Perforce is a lot easier to use than Git - Actually, would like to think am good with git, but sometimes just wonder how it became so big considering the simple or important things (like check-ins and merges) are so complicated.
check-in isn’t a concept in git because there is no server in git. It’s just a log. The beauty of the merkle tree. It’s just GitHub became popular because people didn’t understand that you can rebase from any remote source.
Delta patches become effortless
> It’s just GitHub became popular because people didn’t understand that you can rebase from any remote source.
If everyone uses it wrong, it was designed wrong. See Q-tips.
git is so agnostic about all this stuff that you can even merge completely disparate, unrelated repositories (for example, my own primary "job" repository, and the linux kernel) into the same on-disk structure. Of course, doing so is useless because none of the commits from one repo has any relevance to the other. But because the commits are identified by true GUIDs, there are no collisions, and both sets of commits can happily exist in a single repo.
It's of almost zero utility, but it does (for me) heighten the beauty and elegance of the concepts behind git (and even, the actual implementation).
I found the ability to merge unrelated repositories very useful when collecting various bits of work I did separately and combining it into a single library.
Oh sorry, yeah, didn't use the right term, meant commit... Still, conceptually in terms of source control, think it's the right term (meaning the process of having your code changes added into the main repo).
... and btw, haven't thought about git in awhile, and am reviewing its concepts. It does really have some nice features - always used to think the staging area was an unused feature (and at least for me, it kind of is), but local branches are super useful, and it seems like all that focus on git being a distributed vcs allows (and requires) a dev to work more locally first in very flexible ways. This is really, really nice. Still not sure all the complexity is worth it...
Know this has been said countless times, but some abstraction layer on top of git (that is actually apart of git itself so that it becomes standard) would have been really nice. Understandably, since git started its life in use by sharp linux-devs, usability probably wasn't at the forefront, but yeah, how many countless hours worldwide would have been saved if git was easier to use? (the dreaded botched friday-night commit and update)
Funny because `git pull` is an abstraction like you talk about. It's a `git fetch` followed by a `git rebase`. Staging or `stashing`, it's all just leaves on the merkle tree. The concept of time/branches/doesn't exist. Only previous hash, current hash. It's simple and people have ADDED complexity to make sense of it (coming from other VCS').
True, the pull command is simple. But it's really the pushing and merging that cause the problems.
And, yeah, don't know if i'd call it simple in the way you're using the word. It's simple in that the tool is very "RISC" in nature instead of CISC, doing smaller simple operations instead of larger more complex ones (except for a few exceptions like pull). But for procedures like pushing your code, it's complex. You have to do a lot of things yourself that other VCS take care of. As you probably are aware of, you need to make sure you do things in the correct order, because the tool won't help you if you make a mistake. Don't think something like source control should be this way (especially when other VCS's don't require this amount of thought).
Just my two cents.
My counter argument is “what is the right order?” And you’ll say “the way work gets done” (paraphrasing) but then I’ll say “that’s exactly how git came to be”.
You are right that it gives you more options and more ways to footgun yourself but that mostly came after the fact. It does need a simplicity wrapper or some better defaults.
Yeah, hopefully, someday someone will write a really good one that becomes universal used.
It's not even baked into Google anymore.
Technically correct, but Piper is API-compatible with Perforce, so while there's presumably no license fee it's still strongly there in spirit.
https://graphite.com/blog/google-perforce-to-piper-migration
It's hardly the same thing at this point. For instance you could say Nginx is API-compatible with Apache, and yes that is correct (Maybe? Let's not split hairs here). But to go as far as saying Apache is state of the art and link to Apache's homepage (like throw2ih020 did) is just like.... no.
I think you think I'm trying to correct/one-up reactordev. I'm not. I agree with him. He's pointing out throw2ih020 is being ridiculous, and I'm pointing out that it's even more ridiculous because it's less than 3.
Is Google a big 3 media/entertainment studio?
While I agree with you, not at Google anymore, I do not consider them the top 3 engine providers that I am referring to in my post. No no, Unreal/Unity/Red/EA/SC it’s all perforce.
Ahhhh. Okay. I didn't realize you meant game studios. Wasn't familiar with that "big 3" term until now. Thanks for clearing that up.
Tech in general, you’re spot on though.
Seems to me that this would be great in general software development for just checking in your dependencies as binaries. No more build-time supply chain attacks with vetted binaries.
Why do you need to lock anything? To prevent someone else from working on it? To prevent duplicative work? That sounds like a management/coordination issue. Not a VCS issue. So you have version A and version B. Why are 2 artists working on the same asset without coordination? Do they not use task tracking or project management? Even if that’s ignored, then one person’s work gets wasted. That happens all the time in text based dev too if it’s lacking coordination.
Should coordination live in the VCS layer? That sounds like it creates more issues than prevents.
Because it's not so cleanly cut. Editing a level might touch several files without you realising, you might move a mesh file which causes 100s of references to update across 100s of files.
For example, you want several people collaborating on a level. Say a level designer, an artist, a lighting person, audio. Usually you split this into several files to allow for the collab but you're bound to hit moments where you need to edit across the board. The level designer moves an area which causes the light, audio, meshes to move. If you don't have a way of knowing someone else is working on those files you're going to step on a lot of toes.
Whether this should live separately from source control is a fair question, but it's just a simple place for it as it's tied directly to the files.
Also, the outcome of a mistake is quite high, it could mean a whole day or two (a week?) of wasted work because you didn't realise someone was also editing the same file.
I totally agree. I worked with Unity for a small indie game and it was sooo hard to use git with my team and merge things like game scenes etc.
It's 2026. Historically the way for large binaries in git was git LFS. Now the way for large binaries in git is just git.
10 months or so ago I believe HN posted "The future of Large Files in Git is Git" : https://tylercipriani.com/blog/2025/08/15/git-lfs/
So is it the future now?
Can this be used for Solidworks models? PDM is so awful
Yeah, engineering is another use case for Perforce, will be interesting to see if Epic pursues that market too. I know UE has some use in industries like architecture and professional simulators.
[flagged]
That sucks, git is so absolutely horrible. It's crazy to me that nobody has made anything better yet. Although I could start that myself and yet have not.
There are some projects:
* https://github.com/jj-vcs/jj
* https://nest.pijul.com/pijul/pijul
Also some older but still kicking alternatives:
* https://darcs.net/
* https://mercurial-scm.org/
With respect, were you around to use any of its predecessors?
I was. I thought, and still think, that svn was much more pleasant to use than git. Alas, I am in the minority.
SVN was more straightforward to use, but that straightforwardness lost a lot in terms of fidelity.
The fact that it was easy to clone a subdirectory was nice; the fact that branches were just subdirectories also was not nice. The fact that tags were mutable since they were also just subdirectories... the fact that every operation you ever did required going to the server (commit, log, checkout, everything) made it a pain if you were on a slow link.
I can't count the number of times I was inspecting SVN history and had to just 'svn log > /tmp/svn.log' so I would have the whole log locally rather than having to hit the server each time I wanted to refine a grep.
git-svn did wonders when we had the code base on subversion
git-svn is good for interacting with an SVN repository using git, but if you want to do a full-on migration there's a lot more to do.
I ended up using KDE's SVN migration tool, which was extremely featureful and versatile, as it was designed for the case of moving a massive, complex SVN repository into multiple separate Git repositories, filtering files/directories, importing tags/branches/etc. Saved me a ton of time.
My favourite part was at the end when we migrated the biggest, most important repo, a lot of the original SVN commits were imported from CVS, meaning we had a direct CVS-to-SVN-to-Git migration keeping all of the original metadata from the CVS commits (such as it was). The data purist in me rejoiced.
https://github.com/svn-all-fast-export/svn2git
Also, for a lot of SVN history running it required such a specific Apache webserver setup that it was at times complex to configure correctly and made it comparatively expensive to find SVN repository hosts. SVN seemed cheap mostly only if you had cheap labor for infrastructure. Very few hosts got to "forge scale" like SourceForge did with CVS or GitHub would eventually/"quickly" do with git.
We are legion ;)
SVN was actually quite decent for game development, definitely more robust and (non-technical-) user-friendly than git+lfs.
(and SVN isn't really compatible with the work-from-home era unfortunately, you really needed a big server on a gigabit LAN)
Yes. Some software is just really bad.
Git is fit for purpose. That purpose is to host a monorepo, with out a lot of 3rd party dependancies, distributed, patch based.
Thats not how everyone else works.
We're all using package managers to help with massive amounts of 3rd party dependancies (why are you version pinning in any place other than your repo, why arent you pulling updates through your repo and reviewing them)
We're reliant on tools like artifactory to make sure those depedancys dont disappear or are not corrupted.
We use yet other tools to manage our binary files (this tool would fix that).
Github, gittea, gitlab, bitbucket... have all added piles of tooling around git, that are grafted on around its short comings.
> It's crazy to me that nobody has made anything better yet.
Because our entire industry has fallen into the rut of "more tools", of stacking turtles (https://en.wikipedia.org/wiki/Turtles_all_the_way_down ) rather than fixing the real issues that hold us back.
> Although I could start that myself and yet have not.
Because unless your a Google or a Linus, no one is going to look twice at your tool for something that is this important. Im not even sure that epic games has the good will, or trust to launch this.
I am going to give them the benefit of the doubt and take a long hard look at it, but my optimism is tempered. But unless it offers a LOT more than git, the extra overhead (lacking IDE support, deployment changes and all the other tooling in GIT's orbit) it isnt going to be a worth while change.
This tool is not for pure source code. It's for videogames. Videogame-specific VCS have been lacking much more than Git has, since the start. As others have said, the biggest problem is undiffable binary files.
Because our entire industry has fallen into the rut of "more tools"... rather than fixing the real issues that hold us back.
This is why I'm not motivated to build something better. I don't think anyone would care.
turns out version control is hard
We did, mercurial just didn't win.
[dead]
Jonathan Blow found it convenient to represent all assets in a large number of text files, to enable merging. For instance he'd have one text file per entity on a map. The game and editor could read either this or the compiled binary version.
Jonathan Blow works with extremely small team sizes relative to the big studios. When you only have a couple people working on a project you don’t need all of the same coordination features.
He and Casey Muratory make a lot of cool instructive content, but their condescending attitude towards the industry always made me thing "Huh, must be really nice working alone and making all the decisions yourself."
I respect his work, but I had to unfollow him on Twitter because he was so condescending to everything and everyone except his loyal fan base.
He’s in a category of influencers who post constantly about gripes and grievances and smug superiority. Some people like that content but I can’t stand it.
I really like hearing about indie development and small teams, but you don’t have to present everything as condescending superiority over the industry. That’s not the part I find interesting.
I think there is an element of audience capture that sets up a self reinforcing feedback loop that drives out the normies and ends up rather cult like.
Is it not also possible that there are an overwhelming number of problems with the big AAA type studios in games right now? I feel right now we're in sort of tale of two cities, because AAA games have turned into barely functioning uninspired parasitically monetized crap, while smaller scale development is in an absolute golden age. And it's likely that LLMs will only add fuel onto this turd burning fire.
In fairness, AAA games are practically over. They have grown too bloated, take too long to develop, and, with the culture wars raging in the background, almost every AAA game steps on at least one landmine and loses a large fraction of its audience before it is even released. Sad, really.
I don't think saying they stepped on landmines is entirely accurate. They jumped into the culture wars headfirst, and simply landed on concrete, as is the typical outcome. Why they chose to do that is an interesting question though, because if they just asked basically any 'normal' person the impact it would have on their sales - the answers would not be ambiguous.
There are real problems, but Jonathan Blow doesn't limit his criticism to those problems. He just indiscriminately criticizes everything. He's turned into the type of critic that is a broken clock that happens to right twice a day.
His criticism isn't limited to AAA game studies. He's a vaccine skeptic and is pretty heavy into far-right influencer garbage. All very surprising if you only know him from his games.
This Tweet is a classic https://x.com/Jonathan_Blow/status/1854708962462982465
Jonathan Blow:
> It doesn't help that all males currently under the age of 40 were raised to be supercucks
I started following him for his gaming work and talks. His influencer content is something else.
Anyone still following him after that needs to fundamentally reevaluate some life decisions.
Maybe he's right on gamedev and wrong on some other things
The interesting thing to consider is that it seems fairly obvious from the outside that big game studios are killing themselves. But somehow, on the inside, there are people, of extensive qualification given the nature of our society, who think they're doing the right thing, and that "we" are wrong.
So it leads to the issue of perspective. Somebody has to be right, and everybody thinks that person is themselves, or they obviously wouldn't think what they do think. So in this case he could well be right on both things, or perhaps neither.
That is what makes it cult like and not a full on cult, there is a lot of truth to what he says. The problems is when the conclusions are extrapolated out to absurdity - it’s hard for me to listen to it. I didn’t take sufficient notes to give a proper recount here and it’s a bit too much work for me to go through it again.
I’ll chip in here and say theres ime a world of difference between the amount of condescension and acerbic noise produced by Blow versus Casey. Casey comes of as grumpy but fundamentally pretty respectful in the stuff I’ve seen him in.
I have only seen Casey's writing in the whole "why is the terminal so slow" debacle, but he was a massive jerk in that. He was right! But still a jerk.
I often feel that while Casey can end up saying you're an idiot, he does not believe you are, only the things you do are. And given the chance, he will happily and warmly engage with "why smart people do idiotic things".
Whereas JB actively thinks you're an idiot, and the things you do are idiotic because of your stupidity, period.
And then they usually back their words by doing things like "you claim that outputting text to a terminal emulator is a PhD level problem, so here I did it in a weekend".
Huge teams are more often than not the sign of bloat and inefficiences.
To be fair, game developers have been rendering text on the GPU for over two decades. I've done it in college a decade ago with bmfont [1] (nowadays the engine does rasterization during import). Whoever thought was making a case with "outputting text to a terminal emulator is a PhD level problem" was really out of their depth and was making a case for unnecessary inefficiencies. Kudos to Casey for proving a point.
1. https://www.angelcode.com/products/bmfont/
I remember when that terminal situation first happened, and my main takeaway wasn't "wow, this Casey guy is a genius" but rather reinforcement of my pre-existing belief that Microsoft is full of incompetent and lazy people. Anyone who has ever dabbled in engine or low-level game dev has implemented basic GPU text rendering at some point.
Yeah, like when Blow claimed he could replace PowerPoint in a weekend and ended up implementing a presentation software that had about 2% of what PowerPoint offers.
Now there's an argument to be made that many don't need the remaining ones but to claim that you 'replaced PowerPoint' for anyone but yourself is ridiculous.
They're good at demos, I give them that.
There's a classic saying along those lines, "everyone is only using 5% of Word. The tricky part is that everyone is using a different 5%"
C++ is the same way and is why while everyone hates 95% of it, no one can agree on a strictly superior language.
Originated (or maybe just popularized) by Joel Spolsky, I think?
Casey was right, though. The windows terminal was (is, it's still there even if you use the new Terminal) atrocious. The performance is so bad, due to going through all the layers it does, which Casey exposed. And it's not even packed with features, pressing up on a new console doesn't bring you a command from history, which Linux terminals and 3rd party Windows ones have been doing for decades, even Powershell does that. The support for colors was also bad, the very limited options for font configuration, and it renders fonts as if it was Win2k... Thankfully, the Windows Terminal solves most of those, and includes tabs and other useful features. Too late for me as I already jumped ship to Linux.
> pressing up on a new console doesn't bring you a command from history, which Linux terminals and 3rd party Windows ones have been doing for decades, even Powershell does that.
I'm wondering if you're confusing Windows terminal with cmd.exe?
Windows terminal is not the shell. It's a terminal emulator. You run a shell inside of the terminal, for example you can run... Powershell.
Command history is a feature of the shell.
Imagine if he had approached it like, e.g. Bruce Dawson though, instead of coming off like an edgelord off his meds.
He did approach it correctly, and then his questions were dismissed because "it needs PhD-level research". If anything, it was clueless and incompetent Microsoft "engineers" who acted as edgelords.
IMO this is just garden variety effects of being a programming influencer. It’s a weird position to be in.
I think being influential just does that to people, with high regularity.
Isn't that kind of the point though? Doing more with less?
He's also the sort of person who I suspect works in a very idiosyncratic way, which is great for him and his mind but probably not everyone else. (This is not a criticism.)
That's fine for database-like meta-data (e.g. game entity properties), but not for images, videos or audio files. Just writing those as hex dumps into text files doesn't make them any easier to merge.
You really can't merge binary data, such as textures, meshes, audio, etc. It doesn't matter if you base64 encode the data and stuff it in a text file: it's a jumble of data (assuming this is the implication of what Blow did).
Tools that support non-destructive editing workflows could in theory provide limited diff and merge capabilities on their internal graphs pf non-destructive operations. E.g. Photoshop could in theory merge two files where unrelated layers have changed. But nobody is actually implementing this because it would be lots and lots of work.
Yeah, it's a cool idea, but for proprietary formats, it's much harder for outsiders to do, and the original companies just don't see it as something they should take care of. My gut says there's too many changes that couldn't be cleanly merged anyway, so they think it's not worth the bother. (And they might be right.)
At least for code, I know people have attempted format-aware, structural diffing for a while. The Lisp communities tried a few times, because s-exps are trivial to turn into trees. None became the standard diff tool, though. However, modern tools like difftastic used tree-sitter to bring a lot of language-aware diffing to the masses.
- https://docs.racket-lang.org/sexp-diff/index.html
- https://github.com/michaelw/mw-diff-sexp
- https://github.com/lambdaisland/deep-diff2
- https://fazzone.github.io/autochrome.html
- https://github.com/Wilfred/difftastic
He uses SVN and specifically has stated that Git isn't suitable for the work he does due to big binaries in source control.
How do you merge changes to a texture, mesh, audio file, etc?
Git LFS has file locking, and no VCS can provide you with the tools for diffing binary assets. I don't see any meaningful difference between Perforce, Diversion or Lore and git + LFS + file locking. Unless there's a meaningful performance impact for large projects (I only work on small / medium projects), the capabilities are the same. However, I get excellent git support for code in any editor, as opposed to Diversion or Lore which have none.
Git LFS for example does not support file chunking. So a single byte change on a large (100s of gigs) file means downloading the whole file again. Lore does chunking of binary files which means faster downloads and better de-dupping on the backend.
Permissions is another thing. Git permissions are done one a per repo basis.
https://epicgames.github.io/lore/explanation/system-design/#...
Also, Lore seems to support checking out only the assets you actually need (on-demand hydration and sparse checkouts), meaning that a level designer can check out just the level that they're working on without having to manually configure a git sparse-checkout (and then not being able to see any of the non-checked-out files).
If this supports dynamic hydration of files, either as they're accessed (like Dropbox with offline files) or by somehow knowing which files need which other files (building a dependency graph) then it could be a massive win both for speed and efficiency of downloads but also for conserving disk space on developer machines.
And since it has API bindings, it's possible that's something that could be built into IDE plugins, so that your editor (Godot, Unity, etc) can know which assets need which other assets and automatically trigger hydration, including when you e.g. try to use a new model/texture/etc in a scene that hasn't used it yet.
I haven't made games for a long time so I can't speak for my experience, only my friends. From what I understand (1) Perforce has decent integrations with the game engine editors my friends work with, so editor support is no factor for them and (2) it has better delta support for the file types they work with - I believe Git LFS mostly uses a generic xdelta diff which is kind of mediocre at everything versus Perforce can understand different file types and be extended to support custom types.
I guess you never worked with anything but git? The devil is in the details, and those details generally suck more in git (or generally: distributed version control systems) than in traditional centralized VCS's.
Also git-lfs is a crutch that breaks more often than it works :/
(I agree though that for small game projects, git is mostly 'good enough', even without lfs).
Git LFS breaks so often that it can't be seen as a serious professional tool tbh. I've had nothing but trouble with it. If it wants to be serious it needs to be built into Git, not added as some after thought.
> Git LFS breaks so often that it can't be seen as a serious professional tool tbh. I've had nothing but trouble with it. If it wants to be serious it needs to be built into Git, not added as some after thought.
At Fortinet we migrated our SVN repositories to git and ran into a ton of issues; developers over the past ten years had done tons of little mistakes that added up, like accidentally checking an entire Windows virtual machine into the repo. In SVN they deleted it and no one ended up caring, but in Git of course it became part of the repo history.
I did a huge amount of work for the migration, 99% of which was analyzing each repo to find out what files/file extensions were overly large, and then either:
1. Filtering them out of the git history completely during import
2. Converting them to LFS objects after the import
The LFS process was certainly better than the other alternatives, which were 'check everything into the git history' or 'remove all the un-diffable binary files and hope that they weren't needed for anything', but it was still not ideal.
Every developer (out of thousands, across multiple countries, timezones, and native languages) had to set their system up properly; if you missed a command, or if you reinstalled your OS and forgot to set up one of the aliases or hooks, then you would end up checking binary blobs into git rather than LFS, or checking out LFS idents rather than the actual files they needed.
We also had the issue of developers fetching code over SSH but LFS files over HTTPS, which would be fine except that we wanted to prevent access to HTTPS from most subnets, so while the developers could use SSH to clone or pull using their 2FA token their client would then make an HTTP request that wouldn't work unless they were on the version control VPN, which.... blah blah blah.
So yeah, it worked better than the alternative, but it did not work _well_ a lot of the time.
There's also the tooling. Game teams have artists and designers where baroque command line incantations are headwinds to their workflow pace.
For the longest time Git tools were really poor. In recent years there's a few ok ones, like Git Fork, though I wouldn't know if those tools scale to the level of a AAA team size repo and not fall over.
Maybe more than a headwind. From my time in AAA here are two things I learned that were true in that environment:
1. Artists hate Perforce.
2. You will never get them to try anything else.
I'm exaggerating, but not by that much. I lost count of the number of artists who are deeply uncomfortable with technology and just manage to learn the bare minimum to do their job, and then will take no more.
So besides everything else Lore needs to nail to be acceptable, they need to make it easy for artists to switch. Maybe when UnrealGameSync grows enough knobs and switches to make it unnecessary for an artist to ever touch P4V, Epic can roll Lore into UGS as an unobtrusive option. And if by then there's good support in Unity, in JetBrains, in Maya, etc., then maybe they'll have something.
don't think it supports branches
it's also tough when you have 1TB of data, over 1mm files and you might want to lock hundreds files in one go
I mean, Git LFS 'supports branches' in that the LFS content identifiers are checked into git as files and Git operated normally; LFS is just a way to replace those content identifiers with the actual content, and then vice-versa when you commit.
I think branching is the one thing that didn't get more complicated with LFS.
No, I meant the locking. Being able to branch a lock so you can edit an asset in a feature branch and then discard it (or force a merge)