I used to play Minecraft with some friends. Eventually, around 1.9, they quit. Their reason was that Minecraft provides a poor multiplayer experience. There is a fundamental difference in how and why I play games compared to how and why they play games. I can play single player for long periods of time and still be enjoying myself. I play because I enjoy games. I stick mostly to single player even in games designed specifically around multiplayer, like Heroes of the Storm and Hearthstone. I actually gave up League of Legends specifically because when I first tried it, it offered a poor single player experience. My friends, however, play games largely for the interpersonal interaction. Minecraft seemed like a good choice, because it offered such a good single player experience. For a while it even seemed like it was good for multiplayer, until we had been playing it together for a while, and my friends started to notice something. We also played Terraria together, and the experience was totally different. In Terraria, we would frequently get together for things. We would build a single central town. We would frequently chat. In Minecraft, even when three or more of us were on, we would rarely work together. We would choose separate locations to build our bases. We would rarely chat. For some reason, even though the games have many similarities, Minecraft did not offer the same interpersonal interaction as Terraria did, and eventually my friends decided that this was such a problem that they did not want to spend their time playing Minecraft at all. Since that time, I have discussed the possible reasons for this with one of those friends. This article is about what we have found and how Minecraft's multiplayer experience could be improved.
The first issue was identified by my friend. He generally prefers to play games with their default music, because it is part of the experience designed by the creators of the game. I generally play for a while with the default music and then turn it off and play my own music instead. Minecraft's music is deliberately designed to be isolating. So he was actually feeling more isolated than I was. This may have even contributed to the decision to choose a more isolated building location. That said though, Minecraft offers no reasons not to claim separate land and build separate bases. When he turned the game music off and played some of his own music, the feeling of isolation diminished significantly. Unfortunately, it still did not remove the actual isolation.
We decided to compare Minecraft to Terraria to find other ways where Minecraft discourages interpersonal interaction while Terraria encourages it, and this produced a lot of discoveries. There are differences in world composition, transportation, character progression, game progression, and content that all contribute to the level of interpersonal interactions in these games. Perhaps the best part is that many of these things could be improved in Minecraft in ways that improve multiplayer without destroying the deliberate feeling of isolation in the single player experience.
Shared Base
The biggest thing we found was NPCs. Terraria has unique NPCs that need houses. The uniqueness of these NPCs is a critical factor here. If they were not unique, everyone could build separate bases and get their own copy of each NPC. Because they are unique though, if everyone built separate bases, the NPCs would be split up between them, and this would make using them horribly inconvenient for everyone. The lack of this single mechanic makes an enormous difference in the isolation of Minecraft.
Minecraft does not necessarily need unique NPCs to fix this issue, and besides, unique NPCs would interfere with the intentional feeling of isolation in the single player game. What it could do, however, is add a server option that will generate a village at spawn and place sufficient light sources or even a wall around it, to make it safer from getting wiped out by mobs before the player is strong enough to protect the village. This would give the players a shared base to start from.
A village at world spawn would provide several advantages that encourage players to build around the village instead of establishing separate bases. One is that it would give players a place to stay the night. If several of the houses had beds, it would be even better. Yes, it would reduce the difficulty of the early game, but given that part of that difficulty is the tedium of waiting out the first few nights and the fact that the early game is generally fairly short, this is not a huge sacrifice. This would, of course, provide players with a motivation to frequent the spawn area, since the villagers generally offer some valuable trades. This makes the spawn a good place to establish a base, for everyone. If the village protection is good but not perfect, that would also offer motivation to stick around to defend the village. These advantages would encourage players to establish a shared base near the spawn village, instead of claiming separate plots of lands and building their bases away from those of other players.
These things together would probably make a bigger difference than anything else. If the players were motivated to establish a common base, every time they returned to base there would be opportunities for interaction. For some, however, this may be too much. If the early game is just too valuable for you to skip, perhaps there could be another option that starts each player with a villager spawn egg already in inventory. This does not interfere with the early game, but it does motivate players to work together to build a village to spawn their villagers in. This even more directly encourages players to build a common base, which still ultimately has all of the benefits of starting with a village at spawn.
Transportation
The second issue was transportation. When a player chats, "I found this awesome thing!" the best you can do is take his or her word for it. In Terraria, if it is close to base, you can use a magic mirror or a recall potion to get there quickly. If it is not, perhaps you have a wormhole potion. The size of even large Terraria worlds is small enough that it is often possible to just walk there within a minute or two. In Minecraft, if you are lucky, the player will post a screenshot some time later. Most of the time though, we don't even bother mentioning the cool things we see or find in chat, because we already know there is no way to convey the full experience.
Minecraft's lack of solid transportation options and enormous world size means that when we are in areas far away from each other, we are isolated by the time it takes to get back together. Even a shared base won't mitigate this, as the only way to get back to spawn fast is death, and when you die, you drop all of the cool loot and equipment you have. Around a year ago, I attempted to make a "magic mirror" mod, that would teleport the player back to spawn when used. Due to a number of technical difficulties, including the programming language and the poor documentation of Forge and Minecraft APIs, I was unable to get it to work correctly, and eventually I was unable to justify all of the time I was spending on it. Someone else has actually successfully made this mod, and it appears to work well, but we should not have to mod Minecraft to get an acceptable multiplayer experience. When a single player is playing the game, the enormous world size is not a huge issue. When playing multiplayer though, Minecraft really needs the magic mirror and other teleportation mechanics more than Terraria does.
There are two critical transportation things Minecraft needs for a good multiplayer experience. The first is a simple way to teleport to spawn, without losing all of your items and experience. The second is a way to teleport to other players. These don't need to be cheap, as long as they are reasonably accessible. The important part is that it is not terribly difficult for players to gain the ability to group up easily.
Ideally, the teleport to spawn item should not require any resources you cannot get in the overworld. The mod I found allows a magic mirror to be constructed from gold, diamond, and lapis, if I recall correctly. This is expensive, but it is accessible by mid-game. Being able to construct a magic mirror device before going into the nether would make it easier to avoid getting stuck there. Personally, I think this is desirable, but if it is not, the magic mirror might not work outside of the overworld. Note, however, that this would seriously hinder its usefulness in multiplayer, because the nether and end would still be very isolating and prevent quality multiplayer interaction. Because the main boss is in The End, the ability for players to group up there is critical for a good multiplayer experience. Keeping players trapped there while others are working to get the necessary gear to be successful is probably a very bad idea in multiplayer as well.
Teleportation to other players is not as easy as in Terraria, because you cannot just locate them on a world map. Instead, it might be better for players to be able to make and distribute special items that will allow other players to teleport to them. When I was attempting the magic mirror, I planned to do this with wall mirrors. A friend might construct a wall mirror, and then he could give it to me. The mirror would retain the identity of the creator though, so when I use the mirror, I would teleport to him. I could then get a wall mirror from each of my friends, put them all in my house, near my spawn, and then if I wanted to go to one of my friends, I could magic mirror home and then use the appropriate wall mirror. Alternatively, potions could be used this way, but it would be impossible to stack potions from one player with those from another, and this would become a serious inventory burden, especially with groups of 5 or 6 players or more. To be feasible, an inventory item based solution would probably need a new UI to select who to teleport to.
There are other general transportation improvements that could also be made though. For example, it is frustrating when you end up spending as much time transporting goods to the surface as you do mining them. The minecart system is still not good enough for general purpose goods transport. It can work, but it is often more work to setup the system than it ultimately saves. Some old rogue-like games had a "Rune of Return" item, scrolls, or spell. When used in a dungeon, this item would teleport you home. It would keep track of where you were though, and when used at home, it would teleport you back to where you were in the dungeon. This could be a valuable addition to the game. Again, it does not need to be cheap, but it should probably be obtainable in the overworld, if it is the only "magic mirror" style item available. It could be an item rarely obtained when enchanting books, or it might be only obtainable from dungeon chests. This would fill the role of a magic mirror item, as well as making cave exploration and mining more accessible.
Now, while it would not bother me, some players might complain about items like this being free to use. If resource management with these items is desirable, there is a simple solution: Make use cost experience. It should not cost a lot, but maybe 10 experience would be a decent cost for a magic mirror, 15 to 20 for a rune of return, and 30 to 40 for a wall mirror. In addition to reducing some of the tedium of travel and making the game more friendly for multiplayer, adding another use for experience would give players more motivation to go fight monsters. As it is, it is trivial to collect 30 to 40 levels if you are good at avoiding death, and there is not really a ton of value in enchanting a lot of stuff with that. Most importantly though, transportation mechanics like this would dramatically improve the multiplayer experience, with or without some usage cost mechanic.
Group Activities
The third issue is lack of motivation to work together and communicate. There are not really any things in Minecraft that require or even encourage team work. Two people mining is no better than just one, because when you have two people, you need twice as much resources. In fact, mining separately improves odds of finding rare ores. There are far more reasons not to work together than to work together.
Terraria's strongest motivation to work together is bosses. Minecraft's lack of many bosses is a major weakness when it comes to multiplayer. Terraria also has events where multiplayer is good, but again, Minecraft does not really have these. "Zombie sieges" were added at one point, then for several releases they never triggered due to a bug, and then they were added back in at 1.8, but they are not really that impressive, as far as events go. The stakes are pretty high, since they can wipe out a village, but there is no indication that they are happening and often villagers are wiped out entirely, sometimes before the player even knows that there is a village nearby. In addition, there is no reward (or conditions) for "beating" the event or even for killing the zombies, so it is generally easier to just build a wall or put blocks in front of the doors and wait out the night. This particular event could be improved by making event zombies drop more loot and be more likely to drop rare loot. If event zombies had twice as many drop chances and the odds of rare loot drops doubled, that would be some pretty good motivation to engage them, especially in the early game, when their iron gear, iron ingot, and plantable crop drops are incredibly valuable. And increased rotten flesh drops would be nice in villages with priests that will buy it. This is only good for the early game. Later on (perhaps when the local difficulty is higher), skeletons might be added to the events. The increased bone and arrow drops would also be a pretty good motivator, while the ranged attacks would increase the challenge. Adding small numbers of spiders and/or creepers even later could also be valuable, as their drops are of significant value in later game potion making.
The place where Terraria really excels in motivation for teamwork, however, is bosses. Minecraft (we are talking about the Java version here) only has two bosses. The Wither is sort of the nether boss, and the Ender Dragon is the End boss. There is no overworld boss though. There is plenty of room for bosses. The overworld's strongholds could have bosses near or in the End portal chamber. Nether fortresses could have bosses. (Though, one might argue that this is exactly what the Wither is. That said, a mini-boss that has a 100% chance of dropping a wither skeleton skull would be a valuable addition.) There is already a model and some data for a giant zombie. This could be a good overworld early-game boss. Perhaps it would spawn after a certain number of zombies were killed, or maybe it could be summoned with a zombie head on top of some particular type of stacked blocks, similar to golems and the Wither. If it moved a little faster than regular zombies, did fairly high attack damage, and had a lot of health, it could easily be a boss worthy of facing multiple players. If it was a summon-only boss, it might also be able to break blocks by kicking them or ramming them (break, but not destroy, allowing players to pick them up during or after the fight). This could make for a pretty intense boss fight that a team of players might work together to prepare for. It should be beatable for 3 to 4 players in leather or iron armor, given the early-game nature of the fight. If the boss was summoned using a zombie head, zombies might have a chance of dropping zombie heads when killed by players during siege events, as they are currently very difficult to obtain early game. Adding more bosses, designed with multiplayer in mind, would really improve the multiplayer game in Minecraft, by providing motivation for teamwork.
Item Diversity
In addition to all of this, there is one other thing that really encourages Terraria players to work together: Class items. Terraria does not have formal classes, and Minecraft does not need them either. Terraria does have informal classes though, established through item bonuses. This would be very easy to add to Minecraft. Right now, there are three major weapon types in Minecraft. It has swords, bows, and potions. There are a few other things you can use as weapons, but they are not anything that could reasonably be turned into informal classes. Imagine an armor enchantment that adds damage to melee attacks. This would not be difficult to implement. If the player hits a mob with a sword, add some damage, depending on the level of the enchantment. (Each level adds half a heart more damage.) If you can get a full set of armor with melee bonuses, you can do rather a lot more damage with a sword. Likewise, enchantments could be added for increasing ranged damage (which may or may not apply to a thrown trident, as well) for players specializing in bows, and enchantments could be added increasing the effectiveness of splash potions (for a twist, this could also boost the effectiveness of splash healing potions, allowing for a "healer" potion subclass). Adding class items to Minecraft could be as easy as adding some enchantments that boost damage from specific weapon types. Why do class items encourage working together in Terraria though? Because if a player is focusing on ranged damage and gets a melee class item, that player is more likely to offer the melee class item to another player that specializes in melee. This also applies to weapons and other things.
Item diversity in general is one of Terraria's strong points when it comes to multiplayer. Even within classes, different items behave differently. One player might prefer a very hard hitting sword that is slow to use, while another might prefer a lighter sword with autoswing that is much faster. As long as they are reasonably comparable, damage per second is not a major factor. The same applies to other classes as well. A magic class sword that makes damaging stars fall on enemies might be perfect for one player, while another prefers a spell that fires lots of tiny crystal shards a short distance. This encourages players to help each other by giving away items they are not interested in but that someone else might like. Item diversity is a huge multiplayer advantage in Terraria.
One place Minecraft could really expand, aside, perhaps, from more interesting items in general, is "accessory" items that provide passive boosts. The Baubles mod shows exactly how this can be done, by adding necklace, ring, and belt slots. Combine this with the class items thing above (as Terraria does), and now you can have a belt of strength that increases melee damage, a ring of dexterity that increases ranged damage, and maybe a necklace of...intelligence?...that boosts splash potions. Or course, accessories could also provide defensive boosts, some of the underwater breathing and swimming enchantments, and so on. Accessories, unlike weapons, could be indestructible (aside from things like lava and cacti), providing enchantments that might normally be found on other gear, at a higher price. For example, the Infinity bow enchantment allows a bow to be used without consuming arrows. When enchanting an accessory, there might be a chance for this enchantment to be given, but only for high end accessories. Perhaps a ring crafted from copper would have no chance of getting the Infinity enchantment, but maybe a diamond ring crafted from gold bars and a diamond would be eligible, at the 30 level enchanting tier. When wearing the ring, the enchantment would apply to any bows the character uses, but the ring would take up a valuable accessory slot that could instead be used for something else. The mod provides one necklace slot, two ring slots, and one belt slot. Enchantments for accessories could include some of the existing weapon and armor enchantments, always applied to whatever appropriate item is in use, as well as increased walking speed, higher jumping, better fishing results, faster mining, shorter mob aggro distances, and more. Accessory items could be crafted and then enchanted, using the existing enchantment mechanics. There might be occasional enchantments that only show up on items in dungeon chests but never when enchanting with the enchanting table. Having a large variety of accessory enchantments would again motivate players to give or trade accessories that not as useful to their play style.
These are not the only places Minecraft could improve its multiplayer experience. Some have suggested that the cave/dungeon generation system needs an overhaul, to make underground exploration more interesting. This would improve both the single player and multiplayer experienced. Some have suggested more mob diversity, which would also improve the game in general. More decorative items, like furniture, would certainly improve the artistic building aspect of the game. The items I have discussed above, however, are where Minecraft could improve the multiplayer game the most. Something to encourage players to work together on a shared base would make a huge difference. Transportation mechanics designed specifically with multiplayer in mind might make the biggest difference of all. Group activities would give players more things to do together. Greater item diversity would encourage cooperative play progression. Most of these are also not terribly difficult. Adding a server option to create a village at the spawn location would be simple. Items for teleportation would be fairly simple to add as well. Adding more group activities would be more work, but improving the one event that already exists, to make it more motivating to engage in would be fairly easy, and it would make a good group activity. Increasing item diversity would be a lot of work, but the low hanging fruit of adding class specific enchantments to the existing enchantments would not be terribly difficult. Minecraft certainly has a ton of room for improvement, but right now, the biggest place it could improve is the multiplayer game. A Minecraft with a better multiplayer game would be far more popular than the current Minecraft, and given how popular the game already is, that is really saying something.
Monday, July 23, 2018
Sunday, July 1, 2018
Cast Off the Training Wheels
Modern programming languages have too many training wheels. In this context, training wheels are language "features" that arbitrarily restrict what the programmer can do with that language. In Java, this includes things like forbidding multiple inheritance and requiring functions to either handle or declare that they can throw errors. In Rust, this includes pointer ownership. Most high level languages deliberately forgo any mechanic for direct memory allocation and manipulation. All of these are "training wheel" features, designed to prevent the programmer from getting into trouble, but they also all take decision making power away from the programmer, and most deny the programmer some very valuable tools. Humans have been programming for well over a century, and we have been programming physically existing machines for over half a century. Isn't it time to take off the training wheels?
Like early bicycles, early programming languages did not have training wheels. Restrictions were typically the result of practical considerations. For example, dynamic typing requires a far more complex compiler, and it requires the program to store types and do frequent type checking. Hard coding types make code faster and easier to compile, and it produces much faster programs. On systems or in applications where performance is important, static typing is the practical choice. As computing systems grew in resources, development time grew in priority, and languages with dynamic typing were invented. Then people started arguing about the meta, and the concept of type safety was born. Something similar happened with multiple inheritance. Object oriented programming was invented to provide a thought model that is easier for the typical human to understand. Inheritance was invented to allow this model to support more abstraction. Multiple inheritance was added to allow for even more abstraction. High levels of abstraction are associated with faster programming (though it neutralizes the benefit of making programming easier for regular people to understand). Then someone pointed out that multiple inheritance can result in ambiguous code, and Java decided it need to solve this by eliminating multiple inheritance and replacing it with a similar, far more limited mechanic, that eliminates the ambiguity possible in multiple inheritance at the cost of a significant chunk of the benefits of multiple inheritance. This was not a decision born from practicality. It was deliberate training wheels, designed to protect programmers from themselves, at the cost of some of their freedom. Rust's ownership system for dynamically allocated memory is another example, where someone noticed some common mistakes programmers make and decided to provide training wheels to make that mistake impossible, again at the cost of eliminating some legitimate and valuable options. It was an ingenious solution, but it comes at the cost of making the language less powerful and the highly skilled programmer less productive. Training wheels are great when they are needed, but to the skilled professional, they often prevent optimum performance. On a bike, training wheels limit the maximum safe speed a cyclist can make a sharp turn. In programming, training wheels lock up some very valuable tools. Now that we have training wheels, we need to learn when it is time to cast them away.
Instead of training wheels, many modern vehicles have warning systems. If you are following too close or are about to hit someone while backing up, an alarm will sound. The car will not prevent you from continuing your action, but it will let you know that what you are doing is not safe. Vehicles that have attempted to second guess the driver have been known to cause accidents or make accidents worse. Sometimes, briefly following too close is necessary to avoid an accident. Sometimes backing into a parked car is better than getting hit by someone driving recklessly. And the driver should always be able to correct any mistake the car might make. In addition, imagine a car that stops at every crosswalk and train crossing, when you can clearly see that no one is coming. Some of these "features" are more like a car that simply won't ever drive across a crosswalk or train crossing, whether it is safe or not. Safety systems are not a bad idea, but machines are still not even close to being able to judge complex situations as well as humans can. Warning systems that allow for human judgment are currently far superior to systems that just refuse to do certain things in the name of safety.
The same applies to programming languages. The language should not try to second guess the programmer. That variable that is assigned a value and then never used might exist to add a critical delay during the initialization of a microcontroller. Warning the programmer that the variable is not being used is useful, but refusing to compile until the variable is used is counterproductive. Warning systems do not have to be built into languages either. In fact, in general, they are better as separate programs, where possible. The C compiler does not need to warn the user about style errors. In fact, it is far more convenient when concerns are separated. Linters warn about style errors, and compilers should stick to syntactical errors that prevent compilation entirely.
The fact is, compilers should not care about best practices, because in software engineering, best practices are heavily dependent on application. Linters are where we are already doing most of the verification of best practices, and they make a good place to verify even more. Linters cannot prevent compilation. The best they can do is issue warnings. But warnings are enough when it comes to best practices in software engineering. Some warnings can and should be ignored. Sometimes the best practice for a particular problem is not the general best practice. Instead of enforcing best practices in language definitions, we should be using external tools to verify best practices and giving programmers the full set of available tools in our programming languages. The best practice in engineering does not include taking away tools that are easy to misuse in ways that are harmful (otherwise, engineers wouldn't have any tools). The best practice is to provide all available tools but include safety protocols that will prevent harm when they are properly followed. In programming that means making languages that allow anything and protocols and linters that will warn users when they are doing things that are potentially harmful.
It's time to take off the training wheels, so programmers can do their jobs better and more efficiently. It's time to take best practices out of our programming languages and put them into linters, so that we will finally have access to full, rich array of programming tools in addition to the ability to use them safely.
Like early bicycles, early programming languages did not have training wheels. Restrictions were typically the result of practical considerations. For example, dynamic typing requires a far more complex compiler, and it requires the program to store types and do frequent type checking. Hard coding types make code faster and easier to compile, and it produces much faster programs. On systems or in applications where performance is important, static typing is the practical choice. As computing systems grew in resources, development time grew in priority, and languages with dynamic typing were invented. Then people started arguing about the meta, and the concept of type safety was born. Something similar happened with multiple inheritance. Object oriented programming was invented to provide a thought model that is easier for the typical human to understand. Inheritance was invented to allow this model to support more abstraction. Multiple inheritance was added to allow for even more abstraction. High levels of abstraction are associated with faster programming (though it neutralizes the benefit of making programming easier for regular people to understand). Then someone pointed out that multiple inheritance can result in ambiguous code, and Java decided it need to solve this by eliminating multiple inheritance and replacing it with a similar, far more limited mechanic, that eliminates the ambiguity possible in multiple inheritance at the cost of a significant chunk of the benefits of multiple inheritance. This was not a decision born from practicality. It was deliberate training wheels, designed to protect programmers from themselves, at the cost of some of their freedom. Rust's ownership system for dynamically allocated memory is another example, where someone noticed some common mistakes programmers make and decided to provide training wheels to make that mistake impossible, again at the cost of eliminating some legitimate and valuable options. It was an ingenious solution, but it comes at the cost of making the language less powerful and the highly skilled programmer less productive. Training wheels are great when they are needed, but to the skilled professional, they often prevent optimum performance. On a bike, training wheels limit the maximum safe speed a cyclist can make a sharp turn. In programming, training wheels lock up some very valuable tools. Now that we have training wheels, we need to learn when it is time to cast them away.
Instead of training wheels, many modern vehicles have warning systems. If you are following too close or are about to hit someone while backing up, an alarm will sound. The car will not prevent you from continuing your action, but it will let you know that what you are doing is not safe. Vehicles that have attempted to second guess the driver have been known to cause accidents or make accidents worse. Sometimes, briefly following too close is necessary to avoid an accident. Sometimes backing into a parked car is better than getting hit by someone driving recklessly. And the driver should always be able to correct any mistake the car might make. In addition, imagine a car that stops at every crosswalk and train crossing, when you can clearly see that no one is coming. Some of these "features" are more like a car that simply won't ever drive across a crosswalk or train crossing, whether it is safe or not. Safety systems are not a bad idea, but machines are still not even close to being able to judge complex situations as well as humans can. Warning systems that allow for human judgment are currently far superior to systems that just refuse to do certain things in the name of safety.
The same applies to programming languages. The language should not try to second guess the programmer. That variable that is assigned a value and then never used might exist to add a critical delay during the initialization of a microcontroller. Warning the programmer that the variable is not being used is useful, but refusing to compile until the variable is used is counterproductive. Warning systems do not have to be built into languages either. In fact, in general, they are better as separate programs, where possible. The C compiler does not need to warn the user about style errors. In fact, it is far more convenient when concerns are separated. Linters warn about style errors, and compilers should stick to syntactical errors that prevent compilation entirely.
The fact is, compilers should not care about best practices, because in software engineering, best practices are heavily dependent on application. Linters are where we are already doing most of the verification of best practices, and they make a good place to verify even more. Linters cannot prevent compilation. The best they can do is issue warnings. But warnings are enough when it comes to best practices in software engineering. Some warnings can and should be ignored. Sometimes the best practice for a particular problem is not the general best practice. Instead of enforcing best practices in language definitions, we should be using external tools to verify best practices and giving programmers the full set of available tools in our programming languages. The best practice in engineering does not include taking away tools that are easy to misuse in ways that are harmful (otherwise, engineers wouldn't have any tools). The best practice is to provide all available tools but include safety protocols that will prevent harm when they are properly followed. In programming that means making languages that allow anything and protocols and linters that will warn users when they are doing things that are potentially harmful.
It's time to take off the training wheels, so programmers can do their jobs better and more efficiently. It's time to take best practices out of our programming languages and put them into linters, so that we will finally have access to full, rich array of programming tools in addition to the ability to use them safely.
Soft Safety in Flexible Programming Languages
For almost half a century now, there has been an ongoing war between type safety and flexibility in programming languages. It is time for that war to end, because Python 3.5 has demonstrated the mechanics for a solution.
The argument goes like this: One side says that static typing it critical to writing correct programs, because without compile time type checking, it is impossible to guarantee type consistency. In practice, this means that serious bugs can be easily missed and come up rarely enough to not get noticed and fixed but often enough to cause serious problems. In addition, type mismatches are frequently a symptom of the programmer not fully understanding the problem. The most strictly and statically typed languages often produce almost or even completely bug free programs, once the program will compile. On the other hand though, dynamic typing provides some extremely powerful advantages. The ability to add attributes and methods to objects dynamically in Python is an example of dynamic typing that has great value. Likewise, the ability to give a collection of unrelated objects methods with the same names and use them as if they were similar, without using some inheritance mechanic to invoke polymorphism, is also a function of Python's dynamic typing. Dynamic typing is essential to meta-programming, which can allow for writing very complex programs in very little code. Static typing advocates assert that static typing is necessary to avoid obvious errors, while dynamic typing advocates point out research showing that skilled programmers do not generally make those errors even without static typing. It is all a trade off, but the consequences are very real. Static typing is significantly less powerful, often making it take significantly longer to program in statically typed languages than dynamically typed languages, but the stronger guarantee of correctness has the potential to cut significant time out of debugging. The fact is, there is no clear consensus whether one or the other is objectively better, but there are definitely cases where the cost of one is significantly higher than the other. This is especially true in cases where meta-programming would be beneficial, because static typing does not allow for meta-programming.
The ideal language would probably be one that can handle both dynamic and static typing. Unfortunately, no such language exists. Yes, it is technically possible to use something like dynamic typing in C and C++, with liberal use of type casting, unions, structs, and void pointers, but it is ugly and time consuming to do this. In languages like Python, one might enforce a sort of static typing by constantly checking types and throwing exceptions when the wrong type is used, but this misses the benefits of compile time checking, and it comes at a very high performance cost. Like dynamic typing in C/C++, it also makes for ugly, time consuming coding.
The important thing about static typing is that there is a compile time type consistency check. Haskell uses implicit typing. In other words, it looks at how things are used, and it guesses their types based on that. Sometimes it is necessary to explicitly specify types, but most of the time it guesses correctly, and type errors are typically a sign of deeper bugs. C and C++ require the programmer to always specify the type when a variable or function is declared. The similarity, however, is that all of these languages verify type consistency at compile time. A dynamically typed language like Python instead keeps track of types during runtime, only verifying types when there are interactions. Strong runtime type consistency checks are called strict or strong typing (not static typing). Lose or weak typing is when a language uses implicit type casting to avoid type mismatches. There is a general consensus that strict or strong typing is better than lose or weak typing, because lose typing frequently leads to unexpected and undesired behaviors (PHP is a widely criticized language, in part due to the negative side effects of loose typing). Aside from JavaScript (which has also faced significant criticism over this), pretty much all common modern languages have fairly strict typing, limiting implicit casting almost exclusively to casts to floats in math where floats and ints are mixed. (Python also allows some list-like objects to be multiplied by scalars, to repeat the list that many times, but this involves no actual type casting.)
Python 3.5 added type annotations to the language. Prior to this, some Python programmers would place comments after initial variable declarations, specifying the type, to help them keep track of types. The creator of Python decided that this kind of informal annotation was valuable enough to some people that it deserved to be formalized. Type annotations were made a formal part of the language in Python 3.5. Note however, that type annotations are nothing more than a new, formal comment syntax for noting types. They don't look like comments, but they function like comments. According to the document introducing type annotations, the language does not and never will enforce them. They are also known as "type hints". They are not intended for runtime type checking (you still have to do this manually if you want/need it), and aside from making sure the syntax is correct, the Python interpreter totally ignores them.
What is the point of formal type annotations that are ignored? How are they better than comments? The reasoning mentioned in the PEP document introducing the feature, is that comments annotating types make the code less readable and can interfere where other comments are needed. Python has been carefully designed to be as self documenting as possible, and adding a formal type annotation syntax helps with this. That said, there is perhaps a more important use for type annotations. The Python interpreter does not and will likely never enforce static typing. All static typing is, however, is a compile time check, to ensure type consistency. In standard Python, creating such a check would require some type inference mechanic, which is complex and difficult to write. With type annotations, however, one could easily write a linter that will use the annotations to determine type and then verify that types remain static throughout the program. There are two important things though. The first thing is that compile time checks do not actually have to happen at compile time. Compile time is merely the latest they can happen. The second thing is, we do not actually need annotations to be part of the language. This merely makes the code more readable. If the Python community had come up with a consistent comment annotation syntax for this, it would have been just as useful as formal annotations for pre-compile static type checking. It turns out that dynamic typing has never been a problem. Anyone with the skill could have developed a comment syntax and written a linter to verify that typing remains static, for any language with dynamic typing. If no one cared enough to do that, then clearly static typing in any particular language was never a priority for them.
There is a massive advantage to this soft type safety though, and that is that it is possible to mix static and dynamic typing. Most programs written in dynamically typed languages only rarely take advantage of the dynamic typing. Most variables keep their initial type for their entire lifetime. Enforcing static typing on most variables is a valuable strategy for catching and eliminating bugs. But, when you need dynamic typing, the cost of working around that in a statically typed language can be enormous. When this happens, the additional debugging time that might be required in a dynamically typed language pales in comparison to the extra time spent working within static typing instead of using meta-programming. So, what if we could make some variables static and some dynamic? This is precisely what a soft type checker in a dynamically type language can allow. With Python's type annotations, we can use annotations on variables where we know the type should never change, but we can also not use annotations where we need dynamic typing to use our time more economically. This provides the advantages of both!
Type safety is not the only place where this might have value though. One of the things that makes C such a powerful language is its freedom to allocate and use memory dynamically and freely. Unfortunately, this freedom also makes it prone to memory leaks and segmentation faults. Rust introduces an ownership mechanic that allows for most of the power available in C, with the minimal restrictions required to ensure memory safety. This mechanic also ensures thread safety. Except, there are a few places where this safety is not necessary or where it would be better for the programmer to deal with the memory safety instead of the language. Like dynamic memory, these places are rare, but they do exist, and when they come up, it can make a huge difference. Could we perhaps implement a kind of soft memory safety for C, just like type annotations allow for soft type safety in Python? I believe so.
C already has a built-in annotation mechanic. Pragmas are used by the C preprocessor as a sort of annotation that tells C to alter its behavior in a particular way. They can be used to silence certain warnings, for example warnings saying that a variable is never used may be irrelevant in embedded systems programming. In GCC, compiler plugins can also use pragmas. For example, pragmas are used to tell OpenMP where and how to use threads in the program. Pragmas that are not used by anything are generally just ignored. It should be fairly simple to define a pragma syntax that can be used to indicate where and how ownership mechanics should be used, and then a linter could easily be made to interpret those pragmas and verify that the ownership mechanic is being used coherently. Of course, the program will compile whether ownership is being violated or not, but if the linter is used consistently and all errors fixed, a C program using this mechanic should be just as memory safe as a Rust program using language enforced ownership. The best part is that if you have some memory where you know memory safety is not an issue or where you need to ensure memory safety manually for whatever reason, you can just indicate that ownership mechanics should not be applied to it. With this, C itself does not have to guarantee memory safety for your program to have a memory safe guarantee. With a little extra syntax, memory safety can be guaranteed before the compiler even touches the code.
In my opinion, soft safety is actually better than hard safety, because it does not eliminate the benefits provided by more flexible and powerful programming languages. It allows us to take the best aspects of very strict languages and apply them to more dynamic and flexible languages, without losing any of the flexibility. What we need is not type, memory, and thread safety built into languages. What we need is syntax and software that can analyze our source code and guarantee exactly the amount of safety we want and need, without eliminating the features that make many languages so powerful. This kind of soft safety could be another step in the evolution of quality for software development.
The argument goes like this: One side says that static typing it critical to writing correct programs, because without compile time type checking, it is impossible to guarantee type consistency. In practice, this means that serious bugs can be easily missed and come up rarely enough to not get noticed and fixed but often enough to cause serious problems. In addition, type mismatches are frequently a symptom of the programmer not fully understanding the problem. The most strictly and statically typed languages often produce almost or even completely bug free programs, once the program will compile. On the other hand though, dynamic typing provides some extremely powerful advantages. The ability to add attributes and methods to objects dynamically in Python is an example of dynamic typing that has great value. Likewise, the ability to give a collection of unrelated objects methods with the same names and use them as if they were similar, without using some inheritance mechanic to invoke polymorphism, is also a function of Python's dynamic typing. Dynamic typing is essential to meta-programming, which can allow for writing very complex programs in very little code. Static typing advocates assert that static typing is necessary to avoid obvious errors, while dynamic typing advocates point out research showing that skilled programmers do not generally make those errors even without static typing. It is all a trade off, but the consequences are very real. Static typing is significantly less powerful, often making it take significantly longer to program in statically typed languages than dynamically typed languages, but the stronger guarantee of correctness has the potential to cut significant time out of debugging. The fact is, there is no clear consensus whether one or the other is objectively better, but there are definitely cases where the cost of one is significantly higher than the other. This is especially true in cases where meta-programming would be beneficial, because static typing does not allow for meta-programming.
The ideal language would probably be one that can handle both dynamic and static typing. Unfortunately, no such language exists. Yes, it is technically possible to use something like dynamic typing in C and C++, with liberal use of type casting, unions, structs, and void pointers, but it is ugly and time consuming to do this. In languages like Python, one might enforce a sort of static typing by constantly checking types and throwing exceptions when the wrong type is used, but this misses the benefits of compile time checking, and it comes at a very high performance cost. Like dynamic typing in C/C++, it also makes for ugly, time consuming coding.
The important thing about static typing is that there is a compile time type consistency check. Haskell uses implicit typing. In other words, it looks at how things are used, and it guesses their types based on that. Sometimes it is necessary to explicitly specify types, but most of the time it guesses correctly, and type errors are typically a sign of deeper bugs. C and C++ require the programmer to always specify the type when a variable or function is declared. The similarity, however, is that all of these languages verify type consistency at compile time. A dynamically typed language like Python instead keeps track of types during runtime, only verifying types when there are interactions. Strong runtime type consistency checks are called strict or strong typing (not static typing). Lose or weak typing is when a language uses implicit type casting to avoid type mismatches. There is a general consensus that strict or strong typing is better than lose or weak typing, because lose typing frequently leads to unexpected and undesired behaviors (PHP is a widely criticized language, in part due to the negative side effects of loose typing). Aside from JavaScript (which has also faced significant criticism over this), pretty much all common modern languages have fairly strict typing, limiting implicit casting almost exclusively to casts to floats in math where floats and ints are mixed. (Python also allows some list-like objects to be multiplied by scalars, to repeat the list that many times, but this involves no actual type casting.)
Python 3.5 added type annotations to the language. Prior to this, some Python programmers would place comments after initial variable declarations, specifying the type, to help them keep track of types. The creator of Python decided that this kind of informal annotation was valuable enough to some people that it deserved to be formalized. Type annotations were made a formal part of the language in Python 3.5. Note however, that type annotations are nothing more than a new, formal comment syntax for noting types. They don't look like comments, but they function like comments. According to the document introducing type annotations, the language does not and never will enforce them. They are also known as "type hints". They are not intended for runtime type checking (you still have to do this manually if you want/need it), and aside from making sure the syntax is correct, the Python interpreter totally ignores them.
What is the point of formal type annotations that are ignored? How are they better than comments? The reasoning mentioned in the PEP document introducing the feature, is that comments annotating types make the code less readable and can interfere where other comments are needed. Python has been carefully designed to be as self documenting as possible, and adding a formal type annotation syntax helps with this. That said, there is perhaps a more important use for type annotations. The Python interpreter does not and will likely never enforce static typing. All static typing is, however, is a compile time check, to ensure type consistency. In standard Python, creating such a check would require some type inference mechanic, which is complex and difficult to write. With type annotations, however, one could easily write a linter that will use the annotations to determine type and then verify that types remain static throughout the program. There are two important things though. The first thing is that compile time checks do not actually have to happen at compile time. Compile time is merely the latest they can happen. The second thing is, we do not actually need annotations to be part of the language. This merely makes the code more readable. If the Python community had come up with a consistent comment annotation syntax for this, it would have been just as useful as formal annotations for pre-compile static type checking. It turns out that dynamic typing has never been a problem. Anyone with the skill could have developed a comment syntax and written a linter to verify that typing remains static, for any language with dynamic typing. If no one cared enough to do that, then clearly static typing in any particular language was never a priority for them.
There is a massive advantage to this soft type safety though, and that is that it is possible to mix static and dynamic typing. Most programs written in dynamically typed languages only rarely take advantage of the dynamic typing. Most variables keep their initial type for their entire lifetime. Enforcing static typing on most variables is a valuable strategy for catching and eliminating bugs. But, when you need dynamic typing, the cost of working around that in a statically typed language can be enormous. When this happens, the additional debugging time that might be required in a dynamically typed language pales in comparison to the extra time spent working within static typing instead of using meta-programming. So, what if we could make some variables static and some dynamic? This is precisely what a soft type checker in a dynamically type language can allow. With Python's type annotations, we can use annotations on variables where we know the type should never change, but we can also not use annotations where we need dynamic typing to use our time more economically. This provides the advantages of both!
Type safety is not the only place where this might have value though. One of the things that makes C such a powerful language is its freedom to allocate and use memory dynamically and freely. Unfortunately, this freedom also makes it prone to memory leaks and segmentation faults. Rust introduces an ownership mechanic that allows for most of the power available in C, with the minimal restrictions required to ensure memory safety. This mechanic also ensures thread safety. Except, there are a few places where this safety is not necessary or where it would be better for the programmer to deal with the memory safety instead of the language. Like dynamic memory, these places are rare, but they do exist, and when they come up, it can make a huge difference. Could we perhaps implement a kind of soft memory safety for C, just like type annotations allow for soft type safety in Python? I believe so.
C already has a built-in annotation mechanic. Pragmas are used by the C preprocessor as a sort of annotation that tells C to alter its behavior in a particular way. They can be used to silence certain warnings, for example warnings saying that a variable is never used may be irrelevant in embedded systems programming. In GCC, compiler plugins can also use pragmas. For example, pragmas are used to tell OpenMP where and how to use threads in the program. Pragmas that are not used by anything are generally just ignored. It should be fairly simple to define a pragma syntax that can be used to indicate where and how ownership mechanics should be used, and then a linter could easily be made to interpret those pragmas and verify that the ownership mechanic is being used coherently. Of course, the program will compile whether ownership is being violated or not, but if the linter is used consistently and all errors fixed, a C program using this mechanic should be just as memory safe as a Rust program using language enforced ownership. The best part is that if you have some memory where you know memory safety is not an issue or where you need to ensure memory safety manually for whatever reason, you can just indicate that ownership mechanics should not be applied to it. With this, C itself does not have to guarantee memory safety for your program to have a memory safe guarantee. With a little extra syntax, memory safety can be guaranteed before the compiler even touches the code.
In my opinion, soft safety is actually better than hard safety, because it does not eliminate the benefits provided by more flexible and powerful programming languages. It allows us to take the best aspects of very strict languages and apply them to more dynamic and flexible languages, without losing any of the flexibility. What we need is not type, memory, and thread safety built into languages. What we need is syntax and software that can analyze our source code and guarantee exactly the amount of safety we want and need, without eliminating the features that make many languages so powerful. This kind of soft safety could be another step in the evolution of quality for software development.
Subscribe to:
Posts (Atom)