MIG

The Shortsightedness of Linus Torvalds#

By Wang Yue

Steve Jobs' Consideration of Mac OS X#

In 1997, Steve Jobs returned and the development of the next-generation operating system was put on the agenda. At that time, open-source software like Linux was gaining popularity. With the development of the internet, companies like Red Hat and VA Linux became successful, and the bubble grew bigger.

Steve Jobs returned to Apple in 1997.

Steve Jobs acknowledged the benefits of Linux and even mentioned in a slide several years later when introducing the underlying Darwin of Mac OS X: Darwin is a system similar to Linux. At that time, the astute Steve Jobs considered the following questions.

First: Maintaining the BSD code in the NeXTSTEP kernel and peripheral tools required a lot of manpower, and the development of various branches of BSD was clearly not as fast as Linux. Many features were missing and needed to be developed by Apple.

Second: Small companies like Apple needed to leverage their strength. Apple's main competitor was Microsoft, and open-source software also targeted Microsoft. If they joined forces, not only could they gain a good reputation (Apple has always claimed to be the largest open-source software company), but they could also gain substantial benefits and put pressure on Microsoft.

Third: And most importantly, collaborating with open-source organizations could promote the development of Mac OS. After all, mature projects like GCC in open-source software could save time and effort for Apple. It would be great to invest some money and get significant benefits.

Therefore, the idea of using the Linux kernel as an important component of Mac OS X was conceived by this great visionary. Apple had previous experience developing Linux, such as collaborating with OSF to port the Mach kernel to PowerPC before Steve Jobs returned (Apple was the largest PowerPC player, and OSF was the largest Mach player), and running Linux as a service on Mach. This system was called MkLinux, which we will mention in subsequent installments because it not only made important contributions to the portability of Linux but also played a significant role in the XNU kernel technology of Mac OS X.

If Linux could be used as an important component of the system and this idea could gain the approval of Linus Torvalds, who had a significant influence in the open-source software community, he could rally a large group of developers to join Apple. This was the powerful outcome that Apple wanted to see. With this guiding idea, he had his secretary send an email to Linus Torvalds, asking if he had one to two hours to meet with Steve Jobs. Linus Torvalds, who was unaware of the situation, was quite happy to receive the email because it was his first opportunity to visit Silicon Valley.

Linus Torvalds

An Unfruitful Meeting#

Finally, this rare guest arrived at Apple's headquarters, Infinity Loop. Steve Jobs personally received him, and Avie Tevanian, the former NeXT technical director (whose story we will mention in the future), also attended the meeting. Needless to say, the topic of discussion was the still unknown Mac OS X. The discussion was not formal, but Linus Torvalds' rebellious personality led the negotiations to a deadlock.

Naturally, Steve Jobs brought up the theory he presented in his speech at MacWorld when he returned to Apple in 1997. Although Apple was in decline, it was still a remarkable company. There were only two real players in the desktop field worldwide: Apple and Microsoft. Together, they constituted 100% of the desktop user base. So, Linus, why don't you join us? If you join us and let us run Mac on Linux, a large number of desktop users will become Linux users. The future will be bright!

At that time, Linus Torvalds was quite influential. Many big companies like IBM and Red Hat were seeking his attention. He was a celebrity among entrepreneurs and didn't even consider companies like Apple. As a revolutionary in open-source software, he believed that Linux had more potential users than Apple. He always believed that, based on the current development trend of open-source software, he would soon have a share in the desktop field. And with his eccentric personality, even if he could dominate the desktop field, he would still act as if he didn't care about it. So, in reality, Steve Jobs' opening statement failed.

Then, Avie Tevanian introduced the entire plan to Linus Torvalds. They wanted to merge the Mach and Linux kernels as the foundation of Mac OS X. I suspect Linus Torvalds misunderstood (because Avie Tevanian realized early on that hybrid kernels had clear advantages over microkernels). He thought Apple wanted to use Linux as a service running on Mach (although personally, I believe that even if Mach and Linux were merged into a hybrid kernel, Linus Torvalds, with his rebellious personality, would still not accept it). This reminded him of his previous debate with Professor Tanenbaum. And he also knew that the failed project Taligent, which was a collaboration between Apple and IBM, was based on Mach.

Torvalds and Tanenbaum, who had debated the advantages and disadvantages of microkernels.

Linus Torvalds had his own views on microkernels, which he had expressed in different places before. If we remove the sensitive words from the debate about microkernels, it can be summarized into two aspects.

On one hand, designing a microkernel and its related services could lead to various design disasters. As early as the late 1980s, GNU/Hurd considered trying to write a series of Unix services on Mach, but they could never figure out whether these services should send messages to other services first or consider other solutions. So, as of 2011 when I wrote this article, the Hurd project was still in a semi-dead state.

On the other hand, the efficiency of a microkernel cannot compare to that of a traditional kernel. The simplest system call involves communication between a series of low-level services. Many researchers have tried to improve the efficiency of microkernels, resulting in even more complexity. Many research results that could improve the efficiency of microkernels have already been implemented in the Mach project. In Linus Torvalds' view, this made Mach a very complex project with mediocre efficiency.

During the meeting, Avie Tevanian, who was sitting on the side, was actually one of the earliest developers of Mach. He enthusiastically explained the blueprint of the Mac OS X system to Linus. But Linus was already impatient. For example, in Mac OS X, there was a compatibility layer that allowed users to use classic Mac OS programs. This technology was very similar to Wine, which allows Windows programs to run on Unix systems today. Apple's consideration at that time was that because the old Mac OS did not consider issues like memory protection when designing the API, this layer had to be abandoned. All new programs in Mac OS X had to adopt the more advanced API from NeXT (according to my research, there was no concept of Carbon at that time, and in fact, Carbon was incompatible with the classic Mac OS in terms of both API and ABI). And it was not possible to quickly rewrite and migrate existing software to Mac OS X in the short term. Therefore, if users needed to use third-party applications from the old Mac OS, they could use this compatibility layer provided by Apple. However, due to the aforementioned reasons, the old programs did not enjoy the same benefits as the new programs because when the emulator itself ran multiple old Mac OS tasks, just like the old Mac OS, there was actually only one process without memory protection. The benefits of doing this were obvious because, on the one hand, the old programs could still be used when Mac OS X was first released, and on the other hand, Apple clearly distinguished itself from the old technology, forcing developers to use the new technology. The technical reasons were the most important.

Avie Tevanian

But Linus Torvalds found this seemingly correct technology strange. He assumed that it was entirely possible to run multiple different emulator processes to perform different tasks, allowing each task to enjoy memory protection. This romantic idea made him despise the intelligence of Apple employees. However, when I used early versions of Mac OS X, I found that Linus Torvalds' idea was completely impractical. Because this emulator layer itself required a considerable amount of memory and CPU, running multiple emulator processes in the late 1990s when processor speeds were not as fast as today and memory was extremely precious was equivalent to causing trouble for oneself.

Steve Jobs, considering that Linus Torvalds was a leader in open-source software, continued to talk about open-source as a topic, appealing to his emotions and reasoning with him. He told Linus Torvalds that after we create this system, all Unix layers (excluding the graphical interface layer) will be open-source. So, in fact, if you join us, you will also be contributing to open-source! However, due to his long experience in the open-source community, Linus Torvalds was not moved by this. He believed that no one would want to use a system where the underlying layer is open-source but the graphical interface is not. So, users like me were "represented."

Mac OS X and Linux Part Ways#

In short, this meeting completely fell apart. The two sides looked at the problem from different perspectives, and both Steve Jobs and Linus Torvalds were strong-willed individuals who believed in their own opinions. Their technical and business considerations were different, so the discussion was like talking to a brick wall. This discussion led Apple to abandon Linux and instead adopt FreeBSD technology. In 2001, Jordan Hubbard, the founder and leader of FreeBSD, was appointed as the manager of the BSD technology group at Apple and later became the Director of Unix Technology. As for the direction of Apple's kernel technology afterward, we will discuss it in the next installment.

Jordan Hubbard

In my opinion, the breakdown of the negotiations between Apple and Linus Torvalds, from today's perspective, was due to Linus Torvalds' arrogance and shortsightedness. He did not understand the importance of respecting the opinions of other developers and constantly resisted them. This included the later debate about C++. After the release of Mac OS X, Linus Torvalds repeatedly mocked its outdated technology and said that he had anticipated these issues during his meeting with Steve Jobs. Only recently has he become somewhat more mature and started to soften his views on Mac OS X. However, he still criticizes the Mac file system as garbage (in fact, Linux's file system is not much better; at least Apple had experimented with ZFS for a while). This personality ultimately led to Linus Torvalds not even getting a taste of the success of Mac OS X and iOS when they became popular.

In fact, this was also good for Apple. Apple's focus is on profit, not on unnecessary trouble. Even open-source software is driven by profit. Apple encountered many open-source projects that thought highly of themselves but had poor code quality, such as the GCC compiler project¹. Although they invested a lot of money, it initially solved some problems, but over time, these project team members always clashed with Apple and used their position in the open-source world to intimidate them. In the end, Apple couldn't stand the attitudes, agreements, and code quality of these project team members and felt that it was more convenient to build their own solutions. Therefore, Apple promoted grand projects like LLVM and made it the most advanced open-source software technology in just a few years. This was like a resounding slap to the Linux and GCC teams.

This article was written by Wang Yue, a graduate student in the Computer Science Department at the University of Pennsylvania, a well-known TeX developer in China, and an obscure OpenFOAM developer.

Wang Yue is one of the people I admire most. This series of articles has been published for over ten years, and there is hardly a complete version available on the internet. I hope to further improve it based on this foundation and make it known to more people.

References: