The author misses the point when he says this might help Java regain share in the space of DevOps tools which is currently mostly Go.
The problem is not artifact size but the JVM slow start-up time. This is made worse by almost all Java frameworks that by definition do all their own stuff before passing control to your real app code.
Such executables are not something you would put in a loop in a shell one liner, and that is the space for command line utilities Go has occupied successfully.
But frameworks (and runtimes for things like Clojure) absolutely do. It would be possible to write a fast-starting framework for Java, but i suspect nobody has because there's no demand for it, because nobody who needs fast startup uses Java, because it doesn't start up fast!
That said, i think the real reasons Go has done so well in devops are (1) it's easy to pick up for devopsists coming from Perl/Ruby/etc backgrounds, (2) static binaries are easier to deploy than a jar plus a JVM (not massively easier - but easier enough), (3) a focus on systemsy stuff in the standard library and community, and (4) sheer snowballing momentum, in that Go has become the default choice for stuff like that.
That's actually pretty fast. Starting the JVM, grabbing the memory for the heap, and then exiting takes one tenth of a second.
Things like this are why all the "Java isn't slow" articles miss the point completely --- 100ms to do nothing useful, on a presumably quite fast machine, is ridiculous.
For comparison, 100ms is roughly the time it takes to grep an 18MB file:
> Things like this are why all the "Java isn't slow" articles miss the point completely --- 100ms to do nothing useful, on a presumably quite fast machine, is ridiculous.
"Ridiculous" is like "unprofessional", it's what you say when you don't have a real argument. Yes, ha ha, 100ms, very droll. What does it matter in practice?
In fact, JVM doesn't start slow. The fact is most Java web apps start slow. The more libraries a Java web app depends, the slower it starts.
The current dominant culture of the Java community is very weird. Although Java has a huge standard library, if you don't use a lot of third-party libraries, you will be viewed as a non-professional Java programmer.
Most popular Java tech stack look open sourced, but are backed by lots of money provided by many companies with bad reputations on pure open source culture.
BTW, slow start-up is not the only problem of many Java apps. GUI Java apps often lag for one or more seconds from time to time, which hurts the experiences of Java GUI apps much.
The third problem of Java apps is they often consume much more memory than apps written in other languages, in particular for the long running Java apps. The longer they run, the more memory they will eat.
[edit] the library use way of Java, by putting library jars into your projects, is not good as the way of other languages, by putting library sources into your projects. By putting library sources in your projects, you can view how the libraries are implemented easily. Yes, you can also put Java library sources into your projects, but the main stream culture of Java doesn't recommend to do this. Just look at maven.
[edit 2] just my personal opinion. I don't like the way to use a separated VM to run many apps. This makes it is hard for me to use a different permission setting for each app. Sometimes I want to block a Java app to access network, but I must block the whole Java VM to achieve this.
There is nothing bad about opensource stack backed by big money, in my eyes.
Second, i am surprised to read, that people would still put dependencies/libs into their projects, even in their source code form? I thought, that was one of the reasons, why dependency management was introduced, to get rid of that. One ships a descriptor, what libs were required/used, when the lib has been packaged. And in one's IDE one can view all the dependencies as one wishes, in binary or source code form. I am very confused about your statement.
(though in the final App, one ships all the libs in the packaged form of the app, unless provided by the runtime environment. But only then. Though its optional, one could also download the deps when starting the App)
And third: The longer an App runs, the more memory it eats?
Its like with any other App: if it needs that much memory and it does not release it afterwards, well, then it is a feature or a bug. If it releases the memory, then it is only released within the JVMs memory space. The common default for thag is 4gb per JVM.
Its like the page cache in linux. Once a file has been read into memory, it will usually sit there for a while, unless someone else needs that memory. Thats why on a system with 128gb and a lot of file access, you see free memory going down and down. But its most of the time the page cache eating up that memory and is mostly not a problem. But can be confusing, when one is not familiar with it.
Maybe a misunderstanding. What is the difference between putting a jar as a dependency (pure binary or with sources) or putting its actual content into the project, that uses it?
Hm. Okay. Interesting idea. I would argue, that it defeats the purpose of a shared library, if you have a branched copy of a dependency with your own patches in your own library project. Plus if you work on, say, 10 libraries at the same time, each having its own branched copy with individual patches, oh well, i wouldnt want to do that manually. You would have to send the patches upstream, so that all other libraries can also profit from it. And you blow up the final jar size. And you get in namespace conflicts (maybe modularization solves it) and so on.
If i own the code of a library, then i have a git repository. I have a build server and i always publish the binary and the source jar to an artifact repository.
Now, when i use this library, my IDE knows how to lookup the source jar, so i can always browse the source code. When i see a bug or need a feature, i go to the repository, patch the code and release a new version of that library. And then i bump up the version number of that library in my own project as well.
If i do not own the code, then i ask for a change with a patch (in best case). And use a workaround in the meantime in my project. Earlier or later the updated version of thr library has been shipped (maven repo) and i bump version number as well and remove my patch.
And worst case: owner does not want to change, well, then i fork the code and add it myself, tag the library, release it to my artifact repository and so on. But i barely do that.
In any way, it all relates to my artifact repository (which proxies through official repos such as maven repo) and dependency management from maven, gradle, ant, and so on.
My Go way is similar but I don't maintain such artifact repository things. I directly fork the dependency project on github (or elsewhere), then change the dependency path from something like "others.com/projects/foo" to "github.com/myaccount/foo".
But the fork will only be made if I have confirmed the fork is worthy to make. Before the potential fork, I temporarily modify the code in the local clone of "others.com/projects/foo" directly. If I find my modifications are useless, I will revert the modifications so no forks will be made.
Other processes are like what you described.
Yes, there is still not a perfect dependency management tool in Go world, but there is one official dependency management tool in developing, which will be released alongside with Go 1.10.
> In fact, JVM doesn't start slow. The fact is most Java web apps start slow. The more libraries a Java web app depends, the slower it starts.
Depending on what definition for "slow" you are using it absolutely starts slow.
As mentioned in a cousin comment, the JVM takes ~100ms to get to user code. This makes it impractical for small applications where the user's code takes less than a second to complete. It certainly gets slower with more libraries, but for some applications ~100ms added to startup makes using Java not viable.
Just some hard data to support Java being slow. I ran Hello Worlds in Java, C, Node, Python, and Ruby. Java is the slowest out of all the languages I tested.
time java HelloWorld
Hello, World
real 0m0.170s
user 0m0.148s
sys 0m0.012s
time helloWorld
Hello, World!
real 0m0.028s
user 0m0.004s
sys 0m0.008s
time nodejs hello.js
hello world
real 0m0.136s
user 0m0.104s
sys 0m0.012s
time python hello.py
Hello World
real 0m0.050s
user 0m0.016s
sys 0m0.020s
time ruby hello.rb
Hello World
real 0m0.111s
user 0m0.064s
sys 0m0.028s
> Although Java has a huge standard library, if you don't use a lot of third-party libraries, you will be viewed as a non-professional Java programmer.
The standard library is where modules go to die. The fact that this phrase originates in the Python world tells you that this isn't a Java peculiarity.
> Most popular Java tech stack look open sourced, but are backed by lots of money provided by many companies with bad reputations on pure open source culture.
The sad reality is that those companies are the only organisations that commit serious resources to open-source development, outside a couple of specific niches (web development, unix-like OSes, scientific research up to a point). Those stacks aren't open-sourced in other languages, they just don't exist. And the whole point of open-source is that it benefits everyone regardless of how pure or otherwise the original motives were.
> the library use way of Java, by putting library jars into your projects, is not good as the way of other languages, by putting library sources into your projects. By putting library sources in your projects, you can view how the libraries are implemented easily. Yes, you can also put Java library sources into your projects, but the main stream culture of Java doesn't recommend to do this. Just look at maven.
WTF are you talking about? You declare your maven dependencies and they can be resolved as source or binary. In eclipse I can click through to any library function and see its source immediately. (It's one of the best dependency-management systems going, in any language; there's a single central repository that everyone uses in practice, but it's easy to run your own if you want. It's years ahead of everyone else in terms of package signing. There are multiple independent codebases using the same repositories, not just in theory but in practice).
> [edit 2] just my personal opinion. I don't like the way to use a separated VM to run many apps. This makes it is hard for me to use a different permission setting for each app. Sometimes I want to block a Java app to access network, but I must block the whole Java VM to achieve this.
Again completely normal - Python, Ruby, C#, OCaml... will have exactly the same issue. Get a better firewall.
> You declare your maven dependencies and they can be resolved as source or binary. In eclipse I can click through to any library function and see its source immediately.
Surely, you can view Java dependency sources. It is just that most Java programmers don't care about the sources, they are just care about the jars (the default culture).
In my honest opinion, cross-platform by bytecode has few advantages over cross-platform by source nowadays, on the other hand, cross-platform by bytecode has many inconveniences.
And, (in my taste), single central repository is bad. That's why I don't like Node also. The npm central repository is the source of many Trojans. Many Java and Node developers are not aware of and have no ideas on what they have downloaded through chain dependencies.
> Again completely normal - Python, Ruby, C#, OCaml... will have exactly the same issue. Get a better firewall.
> Surely, you can view Java dependency sources. It is just that most Java programmers don't care about the sources, they are just care about the jars (the default culture).
Not my experience. Actually getting to the source of your dependencies, in practice, is easier in Java - just one click in your IDE - than in any other language. Doesn't that suggest that Java programmers care more about sources than other language users, not less?
> In my honest opinion, cross-platform by bytecode has few advantages over cross-platform by source nowadays, on the other hand, cross-platform by bytecode has many inconveniences.
Well you're entitled to your opinion but you should probably back it up with something more specific if you want to convince anyone else.
> And, (in my taste), single central repository is bad. That's why I don't like Node also. The npm central repository is the source of many Trojans. Many Java and Node developers are not aware of and have no ideas on what they have downloaded through chain dependencies.
You don't have to use a central repository if you don't want to, and some people don't, but it's very convenient to have the option if you want it. Maven central requires PGP signatures on anything published there, so it's very easy to enforce that all the dependencies you depend on come from trusted people if you really want to - as far as I know there's no non-JVM language that does that, certainly not with anything like as large a base of signed libraries available.
> Normal != good.
Agreed, but you started out by claiming that Java's culture is very weird. It isn't.
> Well you're entitled to your opinion but you should probably back it up with something more specific if you want to convince anyone else.
The only benefit of bytecode is to save compiling time. Nowadays, CPU becomes much faster than the time Java was invented, 20 years ago. So the benefit is weak now. Another promoted benefit of bytecode is compile-once-run-anywhere. However, I think this is a hoax. It is not special, for script languages code runs anywhere without compiling.
> Another promoted benefit of bytecode is compile-once-run-anywhere. However, I think this is a hoax.
I was once working on a large project on the JVM: our Fat-JAR was really working on Linux, Mac and Windows. The exact same JAR. And it didn't matter on which system we have built that JAR. This may not be possible for every use case - but I don't think it is a hoax.
> It is not special, for script languages code runs anywhere without compiling.
IMHO scripting languages like Ruby, Python, etc. are quite hard to deploy, especially if they have dependencies. In comparison to languages like Java, Go or Rust.
> The only benefit of bytecode is to save compiling time. Nowadays, CPU becomes much faster than the time Java was invented, 20 years ago. So the benefit is weak now.
In terms of throughput I agree with you. But start-up latency is already Java's weakest point, and having to compile on startup would make it worse.
> Another promoted benefit of bytecode is compile-once-run-anywhere. However, I think this is a hoax. It is not special, for script languages code runs anywhere without compiling.
True as far as it goes, but certain types of code simply aren't practical in scripting languages. You'll never see people implementing e.g. an image format decoder in a scripting language; they'll use bindings to a native-compiled (C) library instead, with all the problems distributing that entails. This is why things like docker are so popular with Python users: in theory Python is write-once-run-anywhere, but in practice most Python codebases end up depending on one or more native modules and need to have the correct version compiled. Whereas Java bytecode is adequate for things like image codecs, and in the real world you really do see large Java codebases that don't use any bindings to (non-JVM) native libraries.
By "DevOps tools", I wasn't thinking in terms of `sed`, `awk`, etc. I was thinking in terms of Consul and Vault. Docker and Kubernetes, etc.
The major pillars of DevOps today are long-running processes, very complex pieces of infrastructure. Which might play into the strengths of the JVM, if the packaging and deployment story simplifies and becomes more competitive with Go.
Sure, "shell one liner" utilities might (?) be best served by native compilation. But DevOps is not merely another word for sysadmin.
It actually starts surprisingly fast. We had a massive Java / Scala codebase at a previous place and one of our engineers wrote a fairly comprehensive set of command line tools (even featuring command completion) entirely in Scala / Java on a 1.7 jvm just on the off chance it would be fast enough.
It actually was totally fine and surprisingly snappy. I’d suggest giving it a shot.
I don’t know how to revert the jvm can’t be used for command line tools because of slow startup times zeitgeist but I don’t think it’s accurate anymore.
Yes, it is a pita, but it is a price to pay for the dependency injection, where many components, services, configs from different projects/libs have to be searched (annotations can hide everywhere), started and then wired together in the order they define 'globally'. And once you change a config property, that tree can change as well.
I hope some work will be done there. One could scan the libs in parallel, caching results, building/shipping a search tree on packaging time and so on.
In pure servlet times you could directly define your starting points, define, which libs to exclude or include for the search and so on.
At least no one has to edit xml files anymore to wire components and impls together.
This statement doesn't necessarily generalize, but I can't say I'm a huge fan of what the Spring-flavoured Dependency Injection ended up looking like around here either. It's like... we've taken on ridiculously long startup times, to turn what would be compile-time errors into run-time errors. Seems like a double-whammy of bad.
Maybe I'm a little overly jaded, but the Java apps I've worked on (which is admittedly only a handful) have all had these giant sprawling messes of dependencies; it feels like there's a lot of code to accomplish very little. The Golang-based services I've worked on feel like the opposite; very little code to accomplish a lot of things, without much ceremony.
Edit: And that's not to say that I'm opposed to dependency injection; rather, I'm quite happy to do constructor-based dependency injection by hand. When that starts to feel painful, I look at the system I'm working on and wonder if it's getting too complicated/has too much coupling/etc.
I do not know golang-services. But in the end, somewhere is the code written and it must be shipped, started, maintained and so on. Someone must also write the drivers for used technologies and must bring them into the golang world.
For us as a little company without access to many developers with affordable pay expectations, it is a big relieve, that a lot of specialist knowledge is bundled together in spring. Such as how to do central configuration, logging, monitoring, failover, loadbalancing, streaming, backpressure support, db connection pooling, message queuing, and so on. And you need special knowledge only when something fails. And when it fails, you can assume, its done the same on each service. It helps a lot. You have then a wider pool of people you can ask to help you to do the actual business coding work.
It reminds me a little on how maven defaults said: code belongs into src/main/java and test below src/test/java and then have a little pom.xml,that describes your project. It has brought sooo much relieve to get rid of so many flavors, how people have structured the code. Not to forget the style of make scripts. You can now open many projects in any IDE without any hassle, you do not have to search forever, where the starting point is and so on. I always forget that. Less work for my little brain, when navigating between projects.
So, even with the dependency hell (i totally agree), i am ok with it, as i seem to forget, what i dont have to do on each individual service anymods. And i have access to a rich ecosystem, that is maintained and supported and people react on issues.
I generally agree! The unfortunate thing for me is that when that stuff breaks in some obscure way, I end up being the one to debug it. Magically having private fields populated during application launch is great, until the junior guys can't figure out why one of them is null sometimes.
Yes i share the pain with you. When it does not work, you still must have a senior or a junior willing to dig into it and become senior after some time as well. Pushing the responsibility away to magic sauce is good and bad. Yup.
Take a look at dagger2. It does compile-time dependency injection through generated .java files, so it wastes no time scanning classes at runtime, and you can look at the quite readable generated source code to see exactly what it's doing.
Well, I figure if that was ported to C it still would take a while to start. I don't think it necessarily demonstrates that the JVM is too slow for command line apps. Just more that Spring takes a while to startup.
That's the thing... I've been porting some of the functionality to Golang, and slicing out many layers of abstraction and baggage at the same time. There's some tradeoffs; e.g. I'm actually writing some SQL queries instead of just declaring interfaces that JPA auto-populates with methods, but overall it just feels like a way lighter and less magical codebase.
I see, how you are writing your go code sounds similar to how we wrote our code on the JVM. e.g. We didn't use Spring or other enterprise frameworks like that and would write pretty much direct SQL with jdbc (after several misadventures in various Scala db-wrappers de-jour).
The problem is not artifact size but the JVM slow start-up time. This is made worse by almost all Java frameworks that by definition do all their own stuff before passing control to your real app code.
Such executables are not something you would put in a loop in a shell one liner, and that is the space for command line utilities Go has occupied successfully.