A while back I wrote an article about password security, discussing common password requirements and how to create better passwords. The takeaway of that article is that common wisdom concerning password security is wrong, and the best passwords are long (16 characters or more) and use memorable made up words. Patreon recently sent out a link to some password advice, which includes more common wisdom on passwords and authentication in general that I want to challenge.
Patreon makes a number of claims about password security. One is that the more random a password the more secure it is. This is actually false. Increasing randomness of a password only makes a difference if it changes classes. Suppose a password uses the birth date of the account owner. This is a highly insecure class of password, because anyone who knows sufficient identifying information about the account owner can gain knowledge of his or her birthday. Name may be sufficient if it is fairly unique, and name and state/country of residence can be sufficient even if the name isn't terribly unique. Name and place of birth may be enough to gain access to birth certificates. Even if this is not enough, an attacker might be able to narrow it down to the birth dates of only two or three people, which is still easy enough. Birthday and other personally identifiable information is one class, which also includes things like names of children and pets, other dates of personal significance, personal catch phrases, and so on. This is an incredibly weak class of password, as those familiar with you or who have access to content you have published will likely have access to the information used in your password. Another class is common words. This category is much larger and thus more secure, though it is not more random, and it is still very insecure. Both of these categories are easy to crack, because the total entropy in the password is less than the entropy of the sum of the characters. Increasing randomness of these classes won't make a significant difference. It does not matter how random the common words or elements of personal information are, passwords in these classes will be approximately equally insecure to all passwords in the class, given that they have about the same number of elements. (In this instance, elements are not characters but rather words or coherent pieces of information.) Going from common words to uncommon words is a solid jump into a more secure category, but entropy is still lower than the sum of the entropy of the individual characters. The impact of increasing randomness is still negligible here. This might be represented as the difference between a phrase made of uncommon words and a series of random uncommon words. If the phrase is significantly more common than the words, increasing randomness will make the password more secure, but if the phrase is less common, it will not make any difference. Security is based on the most common coherent element. A phrase is more coherent than a series of words, but if the phrase is less common than the words, an attacker is far more likely to attack the words than the phrase. Once you get into classes where there is no coherency above the character level though, increasing randomness will have no affect. In other words, a 16 character password composed of made up words is not going to be any more secure than a random password made of characters from the same classes. Thus a 16 character password of made up words made of all lowercase characters is no more secure than 16 randomly selected lowercase characters, but it is many times harder to memorize. Adding additional classes of characters (numbers, special characters, uppercase letters) does increase security, but only negligibly, at the cost of making it even harder to memorize. Increasing randomness only increases the security of a password if it pushes the password into a more secure class. Otherwise it just makes the password harder to remember without having a significant impact on security. Once a password is in a class where strategic attacks either will not work or are not worth the cost, increasing randomness will not increase security even if it does push the password into a theoretically more secure class. Suggesting that increasing randomness increases security of a password is rather irresponsible, because in real life applications doing so makes far less difference than merely increasing the length of the password by a few characters, and making a password less memorable motivates the user write it down, making it far less secure.
Another recommendation of Patreon is to use a password manager. Patreon claims that password managers are highly secure, but this is not actually true. A password manager is no more secure than the authentication it uses, which is typically no more secure than the password a user would make up for another account. Password managers have three advantages. One is that they can generate more secure passwords than the average user would make up, without the user having to memorize them. This cannot make passwords significantly more secure than long passwords of made up words though, because there is no universal way to do that. Even going from all lower case to both cases, number, and symbols increases password entropy by less than 2 bits per character. Adding an additional character or two will have a far larger impact. In short, this is not a significant advantage over just making up a few words for a password. Another advantage is having different passwords for different accounts. This is a significant security advantage, but it comes at a heavy cost. Reusing passwords is a significant security issue. In theory, if one account is compromised, reusing passwords can compromise all accounts with that password. In practice, it is more complicated than that. To leverage this, an attacker would have to learn the usernames for all accounts sharing that password. That might be easy if the compromised account is an email account that the other accounts report to, with emails that mention the account usernames. It might also be easy if the attacker has gained access to your computer, though this is less relevant, because in this case the attacker has already compromised something more valuable than a single account. The attacker could just track your logins directly. Further though, compromising one account is only sufficient to obtain a password if the security on that account is already poor. Any competent organization will use obfuscation techniques to protect passwords, such that even if an attacker did gain access, your password would be in little danger. There are still companies that store passwords in plaintext, but this is becoming far less common, and major companies have pretty much all been scared into using good password security by the massive liabilities doing otherwise would entail. The third benefit of a password manager is not having to memorize passwords. Typically, a password manager has a single password used to access all of the others. The passwords are all encrypted, to maintain security, and they are only decrypted when being used. Tracking passwords for you is quite convenient, but it does not increase security. The only place where password managers improve security more than just using easily memorized made up words is making it easier to avoid password reuse. This is certainly valuable, but as internet security in general improves, the value of this diminishes rapidly. The vast majority of people reuse passwords all over the place. If this was as big of an issue as companies like Patreon make it out to be, large scale compromises would be happening constantly, and every large scale security breech would be followed by large scale financial fraud. The fact that we periodically see large scale compromises without immediate large scale financial fraud suggests that current password security measures are sufficiently mitigating the threat of password reuse. Not to suggest that password reuse is entirely without risk, but rather, the evidence suggests that it is no more risky and perhaps significantly less risky than merely using the weakest allowed password.
Password managers come with a serious downside though. If you use a password manager, that means you do not know any of your passwords. It is easy to forget that security has two goals. One is to protect assets from those who should not have access to them. The other is to grant access to those who should have it. Imagine you have a trusted secretary who memorizes all of your passwords for you. What are the risks involved with that? The risks for a password manager are the same. If your secretary dies, you lose access to your accounts. If your password manager quits working, maybe because your hard drive was corrupted, you will also lose access to your accounts. If your secretary is compromised, all of your passwords and usernames could get leaked. If your password manager is compromised, the same could happen. A password manager is a single point of failure in both cases. A password manager that backs up your passwords to the cloud might be able to avoid the first issue, but that is like your secretary telling someone she trusts your passwords or writing them down and storing them somewhere. This mitigates the problem of losing your passwords at the expense of increasing the attack surface, making them more likely to be obtained by someone who should not have them. This can actually increase the attack surface a lot. Commercial password managers store the password data on commercial servers managed by multiple people. This means that backing up the data to the internet could give a lot of people you have never even met access to your data. True, it may be encrypted, but encryption is inherently reversible. Secure web sites typically hash passwords, which is close to impossible to reverse. A password manager needs to be able to reverse it though, to use the passwords, meaning they have to store the passwords in an inherently less secure form. Password managers can be quite valuable, but suggesting that they significantly increase security is also rather irresponsible, without at least discussing how they do that and where they can reduce security.
The final recommendation that needs some discussion is multi-factor authentication. This legitimately increases security, fairly significantly. Multi-factor authentication uses two or more independent authentication methods. Typically one is a password. Entering the correct password will generally trigger a second step, where some other strategy is used to verify that the person logging in is the person who created the account. The most common form of multi-factor authentication is through text messaging. The user will enter the username and password, and the site will send a text to the user's phone and prompt the user to enter some data included in that text. Another form of multi-factor authentication is the use of a separate device that generates temporary keys, which can then be used for a limited time to prove possession of the device. All common forms of multi-factor authentication rely on the possession of some physical device associated with the user, however this is their weakness. If the user loses the device or the device is damaged, access to the account can be lost. Any form of authentication used to recover an account using this sort of multi-factor authentication is either less secure than the multi-factor authentication, rendering it useless, or it is so complex and time consuming that it denies access to the legitimate owner of the account for a potentially unacceptable length of time. Multi-factor authentication can be a very powerful means of increasing security, but the risks associated with it are quite high.
One of the biggest security recommendations for passwords is to never write them down or to keep them in a highly secure location if they are written down. All three of these recommendations either create motivation to write down passwords or are equivalent to writing down passwords. Making a password more random makes it harder to remember, which is why people write passwords down in the first place. A password manager literally writes down your passwords in a digital form. In multi-factor authentication, your phone or key fob becomes a tool for accessing a password. (The key texted or generated is nothing more than a second password.) The risk with all of these is the same as well. Every single one has a high risk of loss. A random password is difficult to remember thus easy to forget. A password manager relies on the integrity of the system or systems the passwords are stored on. A multi-factor authentication device can easily be lost or destroyed. And of course, password managers increase the attack surface, actually decreasing overall security, compared to a strong password that used only on trusted accounts.
Password security is far more complex than even most internet companies understand. Security should both protect from unauthorized access as well as guarantee authorized access. This is why passwords were chosen as the default form of digital security in the first place. A well memorized password is very difficult to lose or steal, but it can grant very quick access when needed. The problem is that "experts" who are too clever for their own good have been recommending bad password generation practices for decades, and this has lead to security issues that have prompted the creation of products that really should not be necessary for anything short of national security, where denial of access may be vastly favorable to a leak of information to unauthorized parties.
A good password has two properties. One is that it is hard for someone who is not authorized to know it to figure it out. The other is that it is easy for those who are authorized to know it to remember. Security is not just preventing unauthorized access. If that was the case, perfect security could be achieved merely by destroying the assets we want to protect. Providing timely and convenient access to those who are authorized is no less important than preventing unauthorized access. This means that any form of security that significantly hinders or compromises authorized access isn't very secure, even if it does perfectly prevent unauthorized access. A password that is hard to memorize cannot reasonably be considered secure, because it does not provide access the way it should. A tool that makes accessing secured assets significantly more difficult is a compromise of security. Any security strategy that creates a significant risk of denial of access to authorized parties cannot reasonably be considered secure. When assets being protected have extremely high value, it may be worth trading some of the accessibility side of security for protection, but this is not the same as increasing security. It is exchanging security for protection. And this should be a decision made by the owner or owners of the assets, not a decision that the owners are pressured into by second or third parties. Patreon should not be telling its users that they should adopt these security measures. Patreon should be telling its users about these security measures, including the risks associated with them, and then it should leave the decision up to its users, without pressuring them into it. Patreon should also be informing its users of alternatives, for example, making your password longer is a far more effective strategy than making it more random.
Password security is a poorly understood topic, especially among those who have the biggest voice on the topic. It is a lot simpler than many people seem to believe, and many of the strategies that are recommended actually reduce security, either by increasing odds of denial of access or by motivating users to circumvent more legitimate protections.
Thursday, October 3, 2019
Thursday, September 19, 2019
HTTP Request Security
Today I learned some very important security implications of how HTTP requests are formed, and how authentication data is handled. It started with this article: https://thehackernews.com/2019/09/phpmyadmin-csrf-exploit.html
In a nutshell, phpMyAdmin, the primary management tool for MySQL and MariaDB databases, has a serious security flaw that allows attackers to delete entire DB servers with a trivial exploit. It works like this: A webpage is setup with a link or even just a tag that attempts to load an external resource. (The example in the article is an img tag.) If a browser with an open phpMyAdmin session loads the page (in the case of automatically loading the resource) or someone using that browser clicks the link, a command will be sent to the DB to delete the server. It's that simple.
This exploit relies on several vulnerabilities. One vulnerability is that phpMyAdmin uses the same default URL on all systems, and almost no one changes it. This allows an attacker to craft a URL that will work on nearly 100% of phpMyAdmin instances. This is more a problem with the database administrators using phpMyAdmin than the software itself.
The second vulnerability is that phpMyAdmin does not check the referrer field of the HTTP request. If it did that, it would be able to easily weed out HTTP requests that didn't come from the phpMyAdmin instance that is currently open. This would make the exploit significantly more difficult, as spoofing the referrer is not something that can generally be done remotely. The victim of the attack would have to instruct the browser to send inaccurate referrer information for this to be circumvented, though security flaws in browsers might still allow exceptions. This is the obvious mitigation point, but it is not the end of the road. There is an even more fundamental vulnerability here, which does not seem to be widely known, even in the security field.
The third vulnerability is the use of the GET method in handing the request to delete the server. At this point, we actually need to look at the code used for this exploit.
It gets worse though. Merely using a POST request instead of a GET request would make this attack much more difficult. Many sites that allow the injection of GET requests will not allow the injection of any other kind of request, severely limiting an attacker's options. But, given a site that allows this, an attacker could still do the work to implement the exploit, because the most fundamental problem here isn't even in the request type. The request type merely guarantees the problem will exist. The fundamental problem is actually using cookies to transmit the session data.
Cookies are very convenient for web developers, because they automate certain critical processes. By default, HTTP is stateless. This means the server does not know what page you are currently viewing nor what you have viewed in the past. The solution to this is for the client to store the state data, and it does this using cookies. Cookies can give the server some ability to track state as well, for example, whether or not a particular browser is logged into a website. In fact, session management is the most common use of cookies. There are two benefits to this. One is that the server can easily put session authentication data in a cookie instead of having to embed it directly in the HTML. The other is that the server can avoid sending session data multiple times. In theory, this second benefit provides greater security, because it is sending sensitive data fewer times. In practice though, it makes little difference, because the cookie will still be sent with every request. Sending it half the times does not double security. All it does is makes an attacker wait marginally longer to get the session data, a small fraction of the time. In short, the only significant benefit is convenience. The problem with cookies is this: They are sent automatically, regardless of where the request comes from. For example, a POST request that relies on the cookie for authentication could still be used to attack phpMyAdmin, even if it did not allow GET requests to change the DB, because the cookie is sent automatically. A POST request that relies on session data stored in the HTML of the page and sent in the POST data would not be vulnerable to this attack, because an attack coming from an outside web page wouldn't have the authentication data, nor would it have any way to obtain it. This is true of any cookie based session management. The browser automatically sends the cookie with any request to the site the cookie is associated with, which means that even a request to that site from an external site will be authenticated, if an open session exists on the browser being used.
How can this vulnerability be mitigated? Surely you cannot be expected to abandon cookies and keep authentication data in the HTML. Perhaps, but there is one more benefit of cookies. If you open a new tab and navigate to a site where your browser has an open session, if it uses cookies, it will already be logged in. If it uses HTML for storing session data though, you will not be logged in, in the new tab. This can be a serious nuisance. It would certainly avoid the problem we are trying to avoid, but the cost would be far too high to many users. There are a few other potential solutions. Perhaps the best one is to split the session data. Put most of the session data in a cookie, but keep a little bit in the HTML, to be sent with form requests and such. That little bit extra is what authorizes the server to make changes to data. Without it, a request with the cookie will still have the page logged in, but if the request attempted to change something, the server will deny the request and a message will be included on the response page indicating that the requested operation failed because it did not appear to come from an authorized web page. This seems to be the best solution to the problem of cookies, without causing users too much grief.
In short, there are several changes phpMyAdmin needs to be secure against attacks of this nature. First, it needs to check the referrer and deny requests that do not come from its own pages. Second, it needs to apply the HTTP methods correctly, using GET only for requesting data, and using POST and perhaps also PUT and DELETE for changing data. Third, it needs to rely less on cookies for storing authentication data, requiring some kind of authentication embedded in the HTML of its own pages for any request that alters the data or the database. All of these together would be pretty bulletproof, and it would avoid an security vulnerability inherent to using cookies to store session data.
Of course, these strategies should probably be applied to any website that stores sensitive data. None of this is difficult, and none of it introduces much additional complexity. The only potential issue is that it might be a little more difficult to include a special write session key to be sent with POST, PUT, and DELETE requests, if you are using certain frameworks designed to make this easier. Even then though, it should still be possible, and the benefit is certainly worth the cost.
In a nutshell, phpMyAdmin, the primary management tool for MySQL and MariaDB databases, has a serious security flaw that allows attackers to delete entire DB servers with a trivial exploit. It works like this: A webpage is setup with a link or even just a tag that attempts to load an external resource. (The example in the article is an img tag.) If a browser with an open phpMyAdmin session loads the page (in the case of automatically loading the resource) or someone using that browser clicks the link, a command will be sent to the DB to delete the server. It's that simple.
This exploit relies on several vulnerabilities. One vulnerability is that phpMyAdmin uses the same default URL on all systems, and almost no one changes it. This allows an attacker to craft a URL that will work on nearly 100% of phpMyAdmin instances. This is more a problem with the database administrators using phpMyAdmin than the software itself.
The second vulnerability is that phpMyAdmin does not check the referrer field of the HTTP request. If it did that, it would be able to easily weed out HTTP requests that didn't come from the phpMyAdmin instance that is currently open. This would make the exploit significantly more difficult, as spoofing the referrer is not something that can generally be done remotely. The victim of the attack would have to instruct the browser to send inaccurate referrer information for this to be circumvented, though security flaws in browsers might still allow exceptions. This is the obvious mitigation point, but it is not the end of the road. There is an even more fundamental vulnerability here, which does not seem to be widely known, even in the security field.
The third vulnerability is the use of the GET method in handing the request to delete the server. At this point, we actually need to look at the code used for this exploit.
http://server/phpmyadmin/setup/index.php?page=server&mode=remove&id=1This is the URL used in the img tag, in the example from the article linked above. If "server" is replaced with the appropriate domain name or IP address, and you have an open session of phpMyAdmin at that URL in your browser, clicking a link that points to that URL will delete the server. The reason this works is that the session authentication data is stored in a cookie. When the browser loads the above URL, that existing session cookie is sent with it, telling phpMyAdmin that it is a legitimate and authorized request. There are two problems here. The first is the use of a GET request to do something that changes data on the server. Officially, the GET method is only for requests that get data, not for requests that send or change data. The POST method is supposed to be used for requests that send data. The PUT method is supposed to be used for requests that change data. The DELETE method is supposed to be used for requests that delete data. The default method is GET, and simple hyperlinks will always use the GET method. Trivially loading external resources, like images and videos, will also always use the GET method. To use any other method, one must either create a form, where the method used can be specified, or make an AJAX request using Javascript. Injecting forms and Javascript is a lot more difficult than injecting a simple link or resource loading tag. Plenty of web publication sites allow hyperlinks and external images, but they typically filter out Javascript, forms, and anything else that could interfere with the functioning of the website. The problem with the GET method is that an attacker can include all of the command information in the URL itself, which is trivial. In the URL above, we can see that the attacker was able to specify which phpMyAdmin page the request was coming from, the command from that page to execute, and the server id (which will default to 1). If the context and the command were instead expected to be sent in a POST request, an attacker would have to either craft a form or an AJAX request to send the command.
It gets worse though. Merely using a POST request instead of a GET request would make this attack much more difficult. Many sites that allow the injection of GET requests will not allow the injection of any other kind of request, severely limiting an attacker's options. But, given a site that allows this, an attacker could still do the work to implement the exploit, because the most fundamental problem here isn't even in the request type. The request type merely guarantees the problem will exist. The fundamental problem is actually using cookies to transmit the session data.
Cookies are very convenient for web developers, because they automate certain critical processes. By default, HTTP is stateless. This means the server does not know what page you are currently viewing nor what you have viewed in the past. The solution to this is for the client to store the state data, and it does this using cookies. Cookies can give the server some ability to track state as well, for example, whether or not a particular browser is logged into a website. In fact, session management is the most common use of cookies. There are two benefits to this. One is that the server can easily put session authentication data in a cookie instead of having to embed it directly in the HTML. The other is that the server can avoid sending session data multiple times. In theory, this second benefit provides greater security, because it is sending sensitive data fewer times. In practice though, it makes little difference, because the cookie will still be sent with every request. Sending it half the times does not double security. All it does is makes an attacker wait marginally longer to get the session data, a small fraction of the time. In short, the only significant benefit is convenience. The problem with cookies is this: They are sent automatically, regardless of where the request comes from. For example, a POST request that relies on the cookie for authentication could still be used to attack phpMyAdmin, even if it did not allow GET requests to change the DB, because the cookie is sent automatically. A POST request that relies on session data stored in the HTML of the page and sent in the POST data would not be vulnerable to this attack, because an attack coming from an outside web page wouldn't have the authentication data, nor would it have any way to obtain it. This is true of any cookie based session management. The browser automatically sends the cookie with any request to the site the cookie is associated with, which means that even a request to that site from an external site will be authenticated, if an open session exists on the browser being used.
How can this vulnerability be mitigated? Surely you cannot be expected to abandon cookies and keep authentication data in the HTML. Perhaps, but there is one more benefit of cookies. If you open a new tab and navigate to a site where your browser has an open session, if it uses cookies, it will already be logged in. If it uses HTML for storing session data though, you will not be logged in, in the new tab. This can be a serious nuisance. It would certainly avoid the problem we are trying to avoid, but the cost would be far too high to many users. There are a few other potential solutions. Perhaps the best one is to split the session data. Put most of the session data in a cookie, but keep a little bit in the HTML, to be sent with form requests and such. That little bit extra is what authorizes the server to make changes to data. Without it, a request with the cookie will still have the page logged in, but if the request attempted to change something, the server will deny the request and a message will be included on the response page indicating that the requested operation failed because it did not appear to come from an authorized web page. This seems to be the best solution to the problem of cookies, without causing users too much grief.
In short, there are several changes phpMyAdmin needs to be secure against attacks of this nature. First, it needs to check the referrer and deny requests that do not come from its own pages. Second, it needs to apply the HTTP methods correctly, using GET only for requesting data, and using POST and perhaps also PUT and DELETE for changing data. Third, it needs to rely less on cookies for storing authentication data, requiring some kind of authentication embedded in the HTML of its own pages for any request that alters the data or the database. All of these together would be pretty bulletproof, and it would avoid an security vulnerability inherent to using cookies to store session data.
Of course, these strategies should probably be applied to any website that stores sensitive data. None of this is difficult, and none of it introduces much additional complexity. The only potential issue is that it might be a little more difficult to include a special write session key to be sent with POST, PUT, and DELETE requests, if you are using certain frameworks designed to make this easier. Even then though, it should still be possible, and the benefit is certainly worth the cost.
Sunday, July 21, 2019
Why Reusable Software is Bad
I don't actually think reusable software is bad, but I do think excessive reliance on it is bad. Reusable software, in the form of libraries, frameworks, or even clips of code off Stack Overflow, is very useful but also very dangerous, and dependence on reusable code is downright harmful.
I've written about frameworks before. I don't like them. They encourage laziness, and they often fool developers into thinking they are saving time when in reality they are not. They also impact design in ways that a well designed set of tools should never do. A perfect tool is one that never gets a thought during design but can then be used to implement the design flawlessly, without any modifications to accommodate for the idiosyncrasies and limitations of the tool. A framework is supposed to be a full toolkit, but why then do most modern websites include at least three frameworks? And worse, why is it that most websites use exactly the same tools from each one, and neglect exactly the same tools from each one?
Reusable software is dangerous, because it breeds dependence and it enables incompetence. In 2016, this was really brought home, when the creator of a NodeJS library removed his 11 lines of trivial code from NPM, and it broke thousands of websites. All these lines of code did was to pad out the left side of a string with spaces or zeroes, a task so trivial it could be done in only 10 lines of code (the eleventh was merely bookkeeping code that would be unnecessary had users implemented it themselves). In fact, a slight style change would have taken it down to 9 lines of code. There was nothing novel about his implementation either. Any competent developer could have written that code in less than a minute. In fact, most applications could have used a sightly shorter version, with only 8 lines of code, provided they only needed space padding or '0' padding and not both. Why didn't they then? They had become so dependent on reusable code that they did not even consider that they could write it themselves. They did not consider that it could one day disappear, because they did not write it and thus had no special right to continue using it indefinitely. Some may even have lacked the competence to write it themselves, and the use of the library enabled them to remain incompetent.
The decision to use reusable software instead of writing it yourself should always be taken very seriously. Why do you want to use reusable software? Is it because you don't know how to do something yourself? If that is the case, then maybe it would be better to look into it. Not only might it be easier to do it yourself, you might find you can make it fit your needs better that way. If you want to use it, because you think it will take longer to do it yourself, keep in mind that we tend to underestimate how long things will take, and that includes learning how to use someone else's code! It is often faster to create your own implementation of a feature than it is to learn to use a framework or library containing that feature. Unless you are going to use a lot of features of a framework or library, it might actually be faster to just implement what you need yourself. And it is worth learning now: Popularity is never a reason to use a particular piece of reusable code. In this field, things come and go very rapidly. Things that get popular fast often disappear fast as well. If you need to use reusable software, because you know it is too difficult or time consuming to do yourself (OpenGL, for example) or because you need to use a lot of the features it offers and it is a good fit for your application, then it might be worth using. If you just want to learn something new, maybe it would be better to learn something with broader applications and better staying power.
I've written about frameworks before. I don't like them. They encourage laziness, and they often fool developers into thinking they are saving time when in reality they are not. They also impact design in ways that a well designed set of tools should never do. A perfect tool is one that never gets a thought during design but can then be used to implement the design flawlessly, without any modifications to accommodate for the idiosyncrasies and limitations of the tool. A framework is supposed to be a full toolkit, but why then do most modern websites include at least three frameworks? And worse, why is it that most websites use exactly the same tools from each one, and neglect exactly the same tools from each one?
Reusable software is dangerous, because it breeds dependence and it enables incompetence. In 2016, this was really brought home, when the creator of a NodeJS library removed his 11 lines of trivial code from NPM, and it broke thousands of websites. All these lines of code did was to pad out the left side of a string with spaces or zeroes, a task so trivial it could be done in only 10 lines of code (the eleventh was merely bookkeeping code that would be unnecessary had users implemented it themselves). In fact, a slight style change would have taken it down to 9 lines of code. There was nothing novel about his implementation either. Any competent developer could have written that code in less than a minute. In fact, most applications could have used a sightly shorter version, with only 8 lines of code, provided they only needed space padding or '0' padding and not both. Why didn't they then? They had become so dependent on reusable code that they did not even consider that they could write it themselves. They did not consider that it could one day disappear, because they did not write it and thus had no special right to continue using it indefinitely. Some may even have lacked the competence to write it themselves, and the use of the library enabled them to remain incompetent.
The decision to use reusable software instead of writing it yourself should always be taken very seriously. Why do you want to use reusable software? Is it because you don't know how to do something yourself? If that is the case, then maybe it would be better to look into it. Not only might it be easier to do it yourself, you might find you can make it fit your needs better that way. If you want to use it, because you think it will take longer to do it yourself, keep in mind that we tend to underestimate how long things will take, and that includes learning how to use someone else's code! It is often faster to create your own implementation of a feature than it is to learn to use a framework or library containing that feature. Unless you are going to use a lot of features of a framework or library, it might actually be faster to just implement what you need yourself. And it is worth learning now: Popularity is never a reason to use a particular piece of reusable code. In this field, things come and go very rapidly. Things that get popular fast often disappear fast as well. If you need to use reusable software, because you know it is too difficult or time consuming to do yourself (OpenGL, for example) or because you need to use a lot of the features it offers and it is a good fit for your application, then it might be worth using. If you just want to learn something new, maybe it would be better to learn something with broader applications and better staying power.
Why I don't Like JSON
JSON is really popular for passing small to medium amounts of data around the web. It is commonly used in AJAX applications for communication between the client and server. JSON can be really handy, because it can represent objects in a human readable form. This is great for debugging, and makes certain aspects of writing web applications a breeze. Unfortunately, it is also terribly wasteful, and most of the time it is unnecessary.
JSON is not the only communication protocol that has problems. In fact, most popular protocols designed to be human readable have serious problems. The two most common ones are JSON and XML. The primary problem in both of these is the passing of unnecessary metadata. For example, most JSON applications pass data that has a well defined, static format. Metadata does not need to be passed with this kind of data, because the metadata is implied. Imagine passing a C struct or a Java object with metadata about the variable types and names. Perhaps for regular JSON users this seems normal and reasonably, but it is not. The metadata in C-like languages will take up several times the memory, bandwidth, and processing power of the data itself, and this is true even if making it all human readable is not a priority. C structs are a really good example here, because they are one of the most well defined, statically formatted, composite data types one can use. In all but the rarest cases, a struct can be passed across a network or some other interprocess communication media, and as long as the struct definition is identical on both sides, the end product will be identical, without the need for any metadata to consume bandwidth. Adding metadata consumes more memory, and it does not just consume a little bit more memory. A standard integer might be 4 to 8 bytes. If human readability is not important, type metadata can use as little as 1 byte. The attribute name could be anywhere between 4 bytes and 16 bytes, depending on coding style (yes, it could be 1 byte, but then your code is no longer human readable, and that is a serious problem). The metadata will generally fall between two and four times the size of the data itself. If the type metadata is human readable, it will likely be between 3 and 6 bytes, making the metadata between three and five times the size of the data itself. JSON avoids sending much type data, because JavaScript is so loosely typed that the parser can just infer the type from the data, but it still has the attribute names, which are often several times the length of the data, even with the data being in human readable format (which is its own problem). XML is far worse. XML is designed to track metadata about metadata. In fact, it is designed to be as deeply nestable as desired. It can track metadata about metadata about metadata and so on as deeply as one may wish to plunge. XML attributes also add yet another form of metadata on top of the hierarchical metadata. It is so human readable that it is actually quite difficult to read without an editor that will collapse the hierarchy to avoid displaying all of the meaningless metadata all at once. In the vast majority of applications, there is no need to pass metadata. In many cases, having the metadata does not even improve convenience for the programmer. The metadata can be useful for debugging, but well designed debugging and testing tools can abstract that need away quite easily.
The other major problem with JSON, XML, and other human readable protocols is general verbosity. Technically this is also a metadata issue, but it exists at a higher level than attribute names. Structural metadata includes braces, brackets, parenthesis, commas, quotation marks, colons, equal signs, and anything else defining the structure of the data or the relationship between data and metadata. Unsurprisingly, this stuff also takes up a lot of memory, bandwidth, and processing power. XML is the biggest offender here, by far, but JSON is not that far behind. The minimal JSON example looks something like this:
There are cases where passing large amounts of metadata may be appropriate. Most reasons for using protocols like this, however, are actually just laziness. For example, JSON and XML do not care much about order. An object with multiple attributes can be passed with the attributes in any order. This isn't a valid excuse though. It's noting more than nesting a poorly defined protocol inside of another protocol that contains metadata, so you can avoid the consequences of lazy coding. A well defined protocol would define length, order, type, and meaning of each piece of data being sent, eliminating the need to send any metadata. In the case where length cannot be predefined or would be unacceptably inefficient to predefine, the protocol would provide a dedicated place to provide the length metadata, but it would not include any unnecessary metadata. The only place where large amounts of metadata are necessary are places where it is impossible or infeasible to use a well defined protocol. Several examples of this are found in web sites. HTTP and HTML are weakly defined protocols. HTTP has a few somewhat positional elements, but the rest is defined by data not by position or length. HTTP headers must be parsed, they cannot be treated like well defined, statically formatted structs. This is necessary though, because applications of HTTP vary so dramatically. Every server cannot define its own unique protocol and expect every client to understand it, nor can every client define its own unique protocol and expect every server to understand it, and the W3C cannot be expected to create a separate standard for every single web site. Some servers may need to communicate different information from others. Some web sites may need to communicate different information from others. Thus a flexible and dynamic approach is necessary. The same is true of HTML at the presentation layer. Every web pages doesn't need the same organization. CSS must by dynamic, because every web page doesn't need the same style and layout. But unless your web application is going to be communicating with a lot of different systems that might have a lot of different needs, there is no reason to use such a flexible and dynamic protocol. Flexible and dynamic protocols are wasteful and inefficient, and they are far more prone to bugs than compact, well defined, statically formatted protocols. Less well defined protocols are also prone to bloat, because they typically ignore extraneous white space. For the applications where they are needed, this is a useful feature, but for well defined applications, this is wasteful and potentially harmful to clients. Bandwidth, memory, and time are not free, after all.
JSON isn't a useless protocol. Dynamic protocols like JSON and XML have their place. That place, however, is not in applications with well defined communication needs. When the need can be filled by designing a much simpler, much more elegant, and much more efficient protocol, then designing and using such a protocol should be the solution. Only when a simple, elegant, static protocol is impossible or unfeasible should a stock dynamic protocol like JSON or XML be used. And even in those cases, it may still be appropriate to use a simplified version, if the parser you are using can handle it.
The real reason I don't like JSON isn't that JSON is bad. The real reason is that it is used as a default in applications where it really shouldn't be and where it actually wastes time instead of saving it.
JSON is not the only communication protocol that has problems. In fact, most popular protocols designed to be human readable have serious problems. The two most common ones are JSON and XML. The primary problem in both of these is the passing of unnecessary metadata. For example, most JSON applications pass data that has a well defined, static format. Metadata does not need to be passed with this kind of data, because the metadata is implied. Imagine passing a C struct or a Java object with metadata about the variable types and names. Perhaps for regular JSON users this seems normal and reasonably, but it is not. The metadata in C-like languages will take up several times the memory, bandwidth, and processing power of the data itself, and this is true even if making it all human readable is not a priority. C structs are a really good example here, because they are one of the most well defined, statically formatted, composite data types one can use. In all but the rarest cases, a struct can be passed across a network or some other interprocess communication media, and as long as the struct definition is identical on both sides, the end product will be identical, without the need for any metadata to consume bandwidth. Adding metadata consumes more memory, and it does not just consume a little bit more memory. A standard integer might be 4 to 8 bytes. If human readability is not important, type metadata can use as little as 1 byte. The attribute name could be anywhere between 4 bytes and 16 bytes, depending on coding style (yes, it could be 1 byte, but then your code is no longer human readable, and that is a serious problem). The metadata will generally fall between two and four times the size of the data itself. If the type metadata is human readable, it will likely be between 3 and 6 bytes, making the metadata between three and five times the size of the data itself. JSON avoids sending much type data, because JavaScript is so loosely typed that the parser can just infer the type from the data, but it still has the attribute names, which are often several times the length of the data, even with the data being in human readable format (which is its own problem). XML is far worse. XML is designed to track metadata about metadata. In fact, it is designed to be as deeply nestable as desired. It can track metadata about metadata about metadata and so on as deeply as one may wish to plunge. XML attributes also add yet another form of metadata on top of the hierarchical metadata. It is so human readable that it is actually quite difficult to read without an editor that will collapse the hierarchy to avoid displaying all of the meaningless metadata all at once. In the vast majority of applications, there is no need to pass metadata. In many cases, having the metadata does not even improve convenience for the programmer. The metadata can be useful for debugging, but well designed debugging and testing tools can abstract that need away quite easily.
The other major problem with JSON, XML, and other human readable protocols is general verbosity. Technically this is also a metadata issue, but it exists at a higher level than attribute names. Structural metadata includes braces, brackets, parenthesis, commas, quotation marks, colons, equal signs, and anything else defining the structure of the data or the relationship between data and metadata. Unsurprisingly, this stuff also takes up a lot of memory, bandwidth, and processing power. XML is the biggest offender here, by far, but JSON is not that far behind. The minimal JSON example looks something like this:
{"var":12}The data being passed is the number 12. The attribute name is much smaller than typical, and the only structural metadata is the mandatory outer braces, the quotation mares, and the colon. This is 10 bytes total. To parse it, one must find the location of the two braces, find the colon, check for commas to see how many attributes the object has, find the location of the quotes between the opening brace and the colon, compare the characters of the text between the quotes with the desired attribute name, and then read the characters between the colon and the closing brace and convert it to a numerical value. Yes, JavaScript and any JSON library will do this for you. No, this does not made it free. Parsing JSON consumes a lot of processor power. The above process takes nine steps. The steps for comparing the attribute name and parsing the value both take a number of steps equal to their length, and the conversion of the number to a numerical data type requires multiple steps per character. Complex JSON with multiple attributes, arrays, and nested objects gets far more complex. If the value will always be fairly small, this value could be send in a single byte. The client knows what to expect, because it requested the data. The metadata is unnecessarily, because the client already knows the metadata. Wrapping it in a JSON object is wasteful. Send as a single byte, no parsing is necessary. A single byte uses a tenth of the bandwidth and memory. Even if the client does not know exactly what to expect, in the vast majority of cases, a single byte is enough to tell it. That's 256 possible things the client could expect, and few applications have more than that. No, it is not as human readable, but ultimately what is the purpose of writing the application? Is the goal maximum convenience for the developers or is it getting a decent quality product to the consumers? Yes, there is some flexibility here, but using JSON in an application like this is absurd. It is not even lazy, because writing the code to send and process the JSON is more work than sending and receiving one byte of data. How does XML measure up? Let's assume we are using raw XML that is non-compliant, so we can skip things like the doctype line, the xml version line, and such. The above expression, in its simplest form, might be rendered this way:
<var>12</var>Of course, compliant XML would include an xml version line and a doctype line, the above would be enclosed in a root tag of some sort, and the tags might have attributes. Alternatively, the root tag could have an attribute named "var" set to 12. Whatever the case, this non-compliant, simplest form is 13 bytes total, which is 3 bytes longer than the JSON version. Parsing is also a problem, though perhaps simpler than the JSON. First, the opening tag must be parsed, by finding the bracketing greater-than/less-than symbols. The "var" text is then compared with the expected attribute name. The parser must then search for the closing "var" tag. If the parser is a full XML parser, it will also have to put some CPU time into making sure the closing tag belongs to the opening tag and another tag at a different nesting level. If not, all it has to do is verify that the closing tag matches the opening tag. At this point it will already know where the opening tag ends and closing tag begins, so reading the value and converting it to an integer is all that is left. In the best case, we are looking at seven steps with a few sub-steps. In this case, the non-compliant XML takes up 30% more memory and bandwidth than the JSON, but it costs around 20% less processing power to parse. If we enforced standards compliance, the XML would be several times larger and it would take much more CPU time to parse as well. Again, just passing the number 12 as a single byte would be far cheaper, eliminating the parsing and minimizing memory and bandwidth. In terms of natural language, this is like the difference between the answer, "I saw twelve cars" and "Twelve" when someone asks how many cars you saw. The first answer is more complete, but the second answer is entirely appropriate, because the metadata is already included in the context of the question. Similarly, if a client asks the server for the value of a particular variable, it might be more complete to respond with an object containing the context, but doing so is unnecessarily verbose, because the context was already included in the request.
There are cases where passing large amounts of metadata may be appropriate. Most reasons for using protocols like this, however, are actually just laziness. For example, JSON and XML do not care much about order. An object with multiple attributes can be passed with the attributes in any order. This isn't a valid excuse though. It's noting more than nesting a poorly defined protocol inside of another protocol that contains metadata, so you can avoid the consequences of lazy coding. A well defined protocol would define length, order, type, and meaning of each piece of data being sent, eliminating the need to send any metadata. In the case where length cannot be predefined or would be unacceptably inefficient to predefine, the protocol would provide a dedicated place to provide the length metadata, but it would not include any unnecessary metadata. The only place where large amounts of metadata are necessary are places where it is impossible or infeasible to use a well defined protocol. Several examples of this are found in web sites. HTTP and HTML are weakly defined protocols. HTTP has a few somewhat positional elements, but the rest is defined by data not by position or length. HTTP headers must be parsed, they cannot be treated like well defined, statically formatted structs. This is necessary though, because applications of HTTP vary so dramatically. Every server cannot define its own unique protocol and expect every client to understand it, nor can every client define its own unique protocol and expect every server to understand it, and the W3C cannot be expected to create a separate standard for every single web site. Some servers may need to communicate different information from others. Some web sites may need to communicate different information from others. Thus a flexible and dynamic approach is necessary. The same is true of HTML at the presentation layer. Every web pages doesn't need the same organization. CSS must by dynamic, because every web page doesn't need the same style and layout. But unless your web application is going to be communicating with a lot of different systems that might have a lot of different needs, there is no reason to use such a flexible and dynamic protocol. Flexible and dynamic protocols are wasteful and inefficient, and they are far more prone to bugs than compact, well defined, statically formatted protocols. Less well defined protocols are also prone to bloat, because they typically ignore extraneous white space. For the applications where they are needed, this is a useful feature, but for well defined applications, this is wasteful and potentially harmful to clients. Bandwidth, memory, and time are not free, after all.
JSON isn't a useless protocol. Dynamic protocols like JSON and XML have their place. That place, however, is not in applications with well defined communication needs. When the need can be filled by designing a much simpler, much more elegant, and much more efficient protocol, then designing and using such a protocol should be the solution. Only when a simple, elegant, static protocol is impossible or unfeasible should a stock dynamic protocol like JSON or XML be used. And even in those cases, it may still be appropriate to use a simplified version, if the parser you are using can handle it.
The real reason I don't like JSON isn't that JSON is bad. The real reason is that it is used as a default in applications where it really shouldn't be and where it actually wastes time instead of saving it.
Thursday, May 30, 2019
Why I Don't Teach Unity
I teach college level video game design. Every semester some student asks about Unity. Unity is a game development platform. Essentially, it is a program that does a lot of the game development work for you. Unity can do things like handling lighting and shadows in 3D graphics. It can deal with animation for you. It provides access to a decent collection of art assets. In all, it can make video game development significantly easier. In software development though, tools that make things easier always come with a price, and complex general purpose tools can come with a high price.
The first reason I don't teach Unity is that it would defeat the point of the class. There are two parts to the course. One is knowledge. This focuses on design. It covers things like how to design a game that keeps players engaged, and how to design games with specific goals in mind. The other part is application. This is about programming. Students could learn the design knowledge and apply it to games in Unity, but the course would be more of an art course, not a computer science course if I did that.
The second reason I don't teach Unity is that big game companies (that might be interested in hiring my students) don't use it. Yes, there are a few large game companies that have used Unity for maybe one fairly small game. This is not typical though. Large game companies write their games in languages like C++, mostly from scratch. If a student applied for a job at a company like Blizzard and cited my game course as a qualification, if I taught Unity, that student would be treated like a second class applicant. Blizzard is not looking for Unity developers. Blizzard is looking for developers who can write games without the need for a crutch. If I want my course to be a good selling point for getting a job in the game industry, I cannot take the lazy approach and teach students to use a game development platform that does all the heavy lifting for them. They need real experience with real tools.
The third reason is teaching problem solving skills and providing practical experience. While many of my students do have an interest in becoming game developers, most of them won't end up in that industry. Learning to write games using Unity will be worthless to these students. It does not carry over to other applications. It also solves the hardest problems for them, denying them experience that will help build their problem solving skills. Writing games from scratch does provide significant experience that will carry over to other applications, and it provides an excellent opportunity to build problem solving skills.
The fourth reason is that relying heavily on 3rd party tools carries a lot of risk and difficulty. The main problem is that tools designed to be general purpose are rarely ideal for any given specific purpose. Unity is designed to be suitable for a wide variety of games. This means that it is tracking data and doing processing that is probably not necessary for any given game. A game of one genre or style might require something that other games don't really need, but Unity has to provide it to accommodate any game someone might want to make. If there are things Unity does not provide, that will make it harder or even impossible to make games that need to rely on those things. This cuts into efficiency, which means that a game written in Unity will almost certainly perform worse than a game written in C++. For small games, this may make no difference, but for big ones it can limit the market to only users with very high end computers. This is one element of risk. The other element of risk is failure. If Unity has a bug that makes your game not work, you can't just fix it. Even if you had access to Unity's code, it could take months to learn the code, find the bug, and then fix it. This is rare, but it is worth considering, and big game companies do consider it, and they don't use 3rd party game development platforms for these reasons.
These are the top four reasons I don't teach Unity. They do not cover everything though. Unity is mostly used by small indie developers who are either not interested in hiring or who cannot afford to pay even the low wages of a CS graduate. It is certainly a useful platform for hobby game makers who like to tinker. It can even be a good gateway into game development for people without much programming experience. It is not appropriate for a course in a college Computer Science department that wants to produce high quality graduates who have good programming skills, and it is certainly not appropriate for giving students knowledge and experience that might help them get a job in the video game industry. This is an industry that gets a ton of applicants, and even a small edge is worth quite a lot.
Unity is not terribly hard to learn. Like I said, it does all of the heavy lifting for you. Real game programming requires decent math skills. Unity does most of the hard math for you. This makes it trivial to learn compared to real game development. There are plenty of online tutorials and courses for learning to make games in Unity, which is why indie developers with little or no education in computer science can do it. People that want to learn Unity have plenty of resources available to them.
I want to provide the best education I can to my students. Teaching them something that is probably not going to be useful to them does not do that. Teaching them something they can easily learn on their own with a little bit of study online does not do that. Teaching them skills that are useful in a variety of places and knowledge that is difficult to find will provide them with the value they deserve.
The first reason I don't teach Unity is that it would defeat the point of the class. There are two parts to the course. One is knowledge. This focuses on design. It covers things like how to design a game that keeps players engaged, and how to design games with specific goals in mind. The other part is application. This is about programming. Students could learn the design knowledge and apply it to games in Unity, but the course would be more of an art course, not a computer science course if I did that.
The second reason I don't teach Unity is that big game companies (that might be interested in hiring my students) don't use it. Yes, there are a few large game companies that have used Unity for maybe one fairly small game. This is not typical though. Large game companies write their games in languages like C++, mostly from scratch. If a student applied for a job at a company like Blizzard and cited my game course as a qualification, if I taught Unity, that student would be treated like a second class applicant. Blizzard is not looking for Unity developers. Blizzard is looking for developers who can write games without the need for a crutch. If I want my course to be a good selling point for getting a job in the game industry, I cannot take the lazy approach and teach students to use a game development platform that does all the heavy lifting for them. They need real experience with real tools.
The third reason is teaching problem solving skills and providing practical experience. While many of my students do have an interest in becoming game developers, most of them won't end up in that industry. Learning to write games using Unity will be worthless to these students. It does not carry over to other applications. It also solves the hardest problems for them, denying them experience that will help build their problem solving skills. Writing games from scratch does provide significant experience that will carry over to other applications, and it provides an excellent opportunity to build problem solving skills.
The fourth reason is that relying heavily on 3rd party tools carries a lot of risk and difficulty. The main problem is that tools designed to be general purpose are rarely ideal for any given specific purpose. Unity is designed to be suitable for a wide variety of games. This means that it is tracking data and doing processing that is probably not necessary for any given game. A game of one genre or style might require something that other games don't really need, but Unity has to provide it to accommodate any game someone might want to make. If there are things Unity does not provide, that will make it harder or even impossible to make games that need to rely on those things. This cuts into efficiency, which means that a game written in Unity will almost certainly perform worse than a game written in C++. For small games, this may make no difference, but for big ones it can limit the market to only users with very high end computers. This is one element of risk. The other element of risk is failure. If Unity has a bug that makes your game not work, you can't just fix it. Even if you had access to Unity's code, it could take months to learn the code, find the bug, and then fix it. This is rare, but it is worth considering, and big game companies do consider it, and they don't use 3rd party game development platforms for these reasons.
These are the top four reasons I don't teach Unity. They do not cover everything though. Unity is mostly used by small indie developers who are either not interested in hiring or who cannot afford to pay even the low wages of a CS graduate. It is certainly a useful platform for hobby game makers who like to tinker. It can even be a good gateway into game development for people without much programming experience. It is not appropriate for a course in a college Computer Science department that wants to produce high quality graduates who have good programming skills, and it is certainly not appropriate for giving students knowledge and experience that might help them get a job in the video game industry. This is an industry that gets a ton of applicants, and even a small edge is worth quite a lot.
Unity is not terribly hard to learn. Like I said, it does all of the heavy lifting for you. Real game programming requires decent math skills. Unity does most of the hard math for you. This makes it trivial to learn compared to real game development. There are plenty of online tutorials and courses for learning to make games in Unity, which is why indie developers with little or no education in computer science can do it. People that want to learn Unity have plenty of resources available to them.
I want to provide the best education I can to my students. Teaching them something that is probably not going to be useful to them does not do that. Teaching them something they can easily learn on their own with a little bit of study online does not do that. Teaching them skills that are useful in a variety of places and knowledge that is difficult to find will provide them with the value they deserve.
Wednesday, May 29, 2019
Why I Hate Frameworks
While most of the software development world has been rushing headlong into mountains of awesome frameworks over the last decade-and-then-some, I have been quietly reinventing the wheel. At least, that is how a lot of developers often see me. I do not like frameworks. I avoid using frameworks. When I am forced to use frameworks, I do not enjoy it. I find frameworks only cause me pain and suffering.
Reusable code is not a bad idea. I am a big fan of well written libraries that save me significant time and energy. That is not what frameworks are though. I have seen well written frameworks, but the truth is, it tends to be a toss up. Your odds of getting a well written framework are not better than your odds of getting a poorly written one. While I have rarely had to peak into the guts of a framework (a distinct benefit of avoiding them), those that I have had to dive into have not been good. On one occasion, a family member offered me a decent bounty to find and fix a bug in a popular framework. I spent a few hours trying to get familiar with the code of the framework, but in the end, I realized that I would have to charge several times the bounty offered to make it worth my time. Over the course of a few hours, I was unable to trace the path through the code that the program was taking. It was not that the framework was particularly complex. I can handle complex code. It was just poorly written. It might have looked like a nice framework for people who would not have understood what was under the hood regardless of quality, but for an experienced software developer, it was absolute garbage. On another occasion, I was doing some QA work for a corporation, and I needed to get a little bit more information out of their automated testing system. Of course, they were using a reasonably popular framework for the automated testing. It took me days of combing through the framework to figure out where everything was happening, so I could add the critical code to provide the required data. Again, it was not a matter of complexity. The problem was that whoever wrote the code was doing things in the most roundabout way possible. There was a massive amount of coupling. Everything was touching everything else, and consequently, to understand one thing, I had to understand everything. Other frameworks I have used are often buggy, and the generally poor design and code quality make it not worth the effort to try to fix the bugs myself. I have also had a few occasions to look inside general purpose libraries, and I typically find them to be of much higher quality. This is not always the case, but when I discover a bug in a library, I do not feel the same anxiety when considering whether to try to fix it myself or not. So the problem is not that reusable code is just a lousy idea. It is a great idea. The problem is that frameworks specifically tend to be low quality.
There are a lot of reasons I avoid frameworks. The worst is the fact that most frameworks require a lot of effort to do even the simplest things. The theory is that frameworks make everything easier. The reality is that they tend to make hard things significantly easier and easy things much harder. For example, the straw that broke the camel's back (and pushed me over the edge, into writing this article) is the ESP-IDF framework for programming the ESP32 micro-controller. I would not have used the framework voluntarily (I actually prefer manipulating the hardware directly), but it turns out enough of the ESP32 is proprietary, and the FreeRTOS real-time operating system only has the framework implementation available, so I have no other option. The ESP32 is an excellent micro-controller, with built in wifi and a lot of other really nice features for its price. The framework, however, is a disaster. For a previous project, I used I2C on an Arduino, which I programmed entirely in C. Because I did not want to write my own I2C driver, I took Arduino's driver and ported it to C. This was mildly challenging, but despite excessive complexity, Arduino's code was written well enough that it was not too much of a chore. Now I am writing the same program for the ESP32, and the ESP-IDF framework's I2C driver is a nightmare. I do not even have to port it to C. It is already in C. Somehow though, it is actually more work to use than it was to port Arduino's driver to C and use that. The fact is, I2C is trivial. It is not a complicated protocol. (It is not terribly well designed, but it is not complicated either.) The most difficult part of porting the Arduino driver was eliminating all of the unnecessary object orientation. ESP-IDF is written for C, so it does not even have object orientation, and yet it manages to be several times more complex. The fact is, I2C is such a simple protocol that the most difficult part of using it should be setting up the pins on your chip and configuring the clock speed, if your chip has a built in I2C controller (the ESP32 does), but that is not how it works with ESP-IDF. No, setting up the I2C device is fairly simple (though, the documentation does nothing to help, and the provided example code is stupidly complex). The first step is setting up a struct to tell it which pins to use, how to initialize them, what mode to use, and what clock speed to use. The second and third steps are a pair of function calls that really could have been rolled into a single function call. One extra function call is not a disaster though, and maybe it is somewhat justified, if you need to enable and disable I2C frequently. That is where the simplicity ends though. With the ported Arduino I2C library, once you have the thing the initialized, you can call a single function to read and a single function to write. You give the functions the address, the command, and a few other parameters, and you are good to go. With ESP-IDF, I2C reads and writes are major productions. First, you have to tell it that you are starting an I2C transaction. Next you have to tell it to start accepting data for the transaction. Then you have to do a pair of write function calls to tell it what you want to write (it may be possible to roll them into one, but all of the examples show it as two). Then you tell it you are done giving it the transaction data. Do not think it has actually sent the command though. After that, you tell it to send the command. Once that is done, you have to tell it to delete the transaction. In Arduino, this is all a single function call. In ESP-IDF, it is a disaster. I2C is a simple protocol! ESP-IDF manages to take something incredibly simple and turn it into something extremely complicated. This is perhaps the biggest reason I hate frameworks. This is not a one time thing either. Even Arduino's I2C driver is overly complicated to use. In the process of porting it, I eliminated a significant amount of setup cruft caused by using an object oriented approach where it is entirely unnecessary. In my experience, every framework has places where it takes something almost trivial and makes it more complicated than it has to be. Libraries sometimes do this too, for example, most high level languages built in libraries make pipes prohibitively hard to use (one of the few places Python really biffs it), but frameworks do this all over the place. Threading libraries tend to do this as well.
Compared to this, most of my other issues with frameworks are trivial. The worst one is that when a framework has a bug, it is a lot harder to fix than a bug in code you wrote yourself, even if the framework is well written, because you are not familiar with the framework's code. I have legitimately abandoned frameworks because they had one critical bug that would have taken me more time to fix than just writing the code myself, from scratch. Now days, I generally just skip the part where I use a framework in the first place. Another is that, in my experience, frameworks often take more time to use than writing code from scratch anyway, because learning a framework takes time. Unless I am going to use most of the features offered by the framework, it is not worth my time to learn it. I have written AJAX functions from scratch many times instead of using JQuery, because it is actually faster to do that than it is to setup JQuery on my web site and look up how to use its (admittedly high quality) AJAX stuff. Frameworks also consume unnecessary space, leading to memory bloat, performance costs, and unnecessary bandwidth costs, unless you are using almost 100% of the features provided. These things may seem trivial, but when you consider the scale on the server side instead of at the individual client, it adds up surprisingly fast. Frameworks also tend to come and go, especially in web development. This means that the framework you are learning today probably will not be worth much to you in a couple of years. It might help you advance in your current company, but most promotion in the software industry is not internal, and that other company you really want to work for probably does not use the framework you are learning for your current job. This is further aggravated by the fact that there are so many frameworks out there that there really is no point learning one that you do not have an immediate application for. Lastly, and this is really more important than the last few issues, frameworks are almost never as general purpose as libraries. Nearly all frameworks are designed with a fairly narrow set of use cases in mind, but then they are advertised with a focus on one design element that is not closely related to those use cases. For example, responsiveness is a popular feature of web frameworks that dynamically load data. Most such frameworks are designed for point-of-sale and inventory management, and they tend to be janky or even downright lousy for anything else. They do not tell you that you should not use them to make a site that needs good quality real-time interaction though. They do not tell you that the responsiveness comes at a heavy performance cost. So you decide to use a framework like AngularJS, because "responsive" sounds like exactly what you want for your browser game, but halfway through, after you have spent many hours learning the framework, you discover that while it might work well for business applications (if you are willing to tolerate the making-easy-things-excessively-complicated problem that it also has), it is a horrible choice for games, because what they mean by "responsive" is not the same as what you thought it meant. (No, I did not actually try to use AngularJS for a browser game. I did use it for an inventory management system though, where it worked fairly well but where the performance costs occasionally reared their ugly heads in ways that do not matter to an inventory management system but that would make a game unplayable.)
There are a lot of problems with frameworks. Their popularity makes it clear that a lot of people are willing to put up with those problems, for the perceived benefits, but in my experience, most of those benefits are ethereal. Recent research suggests that system and development architectures make a much bigger difference in success rates than development stacks anyway. This means that any benefits gained from using a particular framework will almost certainly be dwarfed by improving team communication and spending more time on quality design work. My personal experience supports this conclusion.
What it all comes down to is that if you really want to use a framework, pick one with features you are going to use, and know what you are getting yourself into. Personally, I find it easier and faster to code the features I want myself, and I find that the time spent finding and learning a suitable framework ultimately costs me more than just writing the code myself. You might say I am reinventing the wheel, but the fact is, when there are a million different kinds of track, you need a million different kinds of wheel if you want all of your trains to run smoothly.
Reusable code is not a bad idea. I am a big fan of well written libraries that save me significant time and energy. That is not what frameworks are though. I have seen well written frameworks, but the truth is, it tends to be a toss up. Your odds of getting a well written framework are not better than your odds of getting a poorly written one. While I have rarely had to peak into the guts of a framework (a distinct benefit of avoiding them), those that I have had to dive into have not been good. On one occasion, a family member offered me a decent bounty to find and fix a bug in a popular framework. I spent a few hours trying to get familiar with the code of the framework, but in the end, I realized that I would have to charge several times the bounty offered to make it worth my time. Over the course of a few hours, I was unable to trace the path through the code that the program was taking. It was not that the framework was particularly complex. I can handle complex code. It was just poorly written. It might have looked like a nice framework for people who would not have understood what was under the hood regardless of quality, but for an experienced software developer, it was absolute garbage. On another occasion, I was doing some QA work for a corporation, and I needed to get a little bit more information out of their automated testing system. Of course, they were using a reasonably popular framework for the automated testing. It took me days of combing through the framework to figure out where everything was happening, so I could add the critical code to provide the required data. Again, it was not a matter of complexity. The problem was that whoever wrote the code was doing things in the most roundabout way possible. There was a massive amount of coupling. Everything was touching everything else, and consequently, to understand one thing, I had to understand everything. Other frameworks I have used are often buggy, and the generally poor design and code quality make it not worth the effort to try to fix the bugs myself. I have also had a few occasions to look inside general purpose libraries, and I typically find them to be of much higher quality. This is not always the case, but when I discover a bug in a library, I do not feel the same anxiety when considering whether to try to fix it myself or not. So the problem is not that reusable code is just a lousy idea. It is a great idea. The problem is that frameworks specifically tend to be low quality.
There are a lot of reasons I avoid frameworks. The worst is the fact that most frameworks require a lot of effort to do even the simplest things. The theory is that frameworks make everything easier. The reality is that they tend to make hard things significantly easier and easy things much harder. For example, the straw that broke the camel's back (and pushed me over the edge, into writing this article) is the ESP-IDF framework for programming the ESP32 micro-controller. I would not have used the framework voluntarily (I actually prefer manipulating the hardware directly), but it turns out enough of the ESP32 is proprietary, and the FreeRTOS real-time operating system only has the framework implementation available, so I have no other option. The ESP32 is an excellent micro-controller, with built in wifi and a lot of other really nice features for its price. The framework, however, is a disaster. For a previous project, I used I2C on an Arduino, which I programmed entirely in C. Because I did not want to write my own I2C driver, I took Arduino's driver and ported it to C. This was mildly challenging, but despite excessive complexity, Arduino's code was written well enough that it was not too much of a chore. Now I am writing the same program for the ESP32, and the ESP-IDF framework's I2C driver is a nightmare. I do not even have to port it to C. It is already in C. Somehow though, it is actually more work to use than it was to port Arduino's driver to C and use that. The fact is, I2C is trivial. It is not a complicated protocol. (It is not terribly well designed, but it is not complicated either.) The most difficult part of porting the Arduino driver was eliminating all of the unnecessary object orientation. ESP-IDF is written for C, so it does not even have object orientation, and yet it manages to be several times more complex. The fact is, I2C is such a simple protocol that the most difficult part of using it should be setting up the pins on your chip and configuring the clock speed, if your chip has a built in I2C controller (the ESP32 does), but that is not how it works with ESP-IDF. No, setting up the I2C device is fairly simple (though, the documentation does nothing to help, and the provided example code is stupidly complex). The first step is setting up a struct to tell it which pins to use, how to initialize them, what mode to use, and what clock speed to use. The second and third steps are a pair of function calls that really could have been rolled into a single function call. One extra function call is not a disaster though, and maybe it is somewhat justified, if you need to enable and disable I2C frequently. That is where the simplicity ends though. With the ported Arduino I2C library, once you have the thing the initialized, you can call a single function to read and a single function to write. You give the functions the address, the command, and a few other parameters, and you are good to go. With ESP-IDF, I2C reads and writes are major productions. First, you have to tell it that you are starting an I2C transaction. Next you have to tell it to start accepting data for the transaction. Then you have to do a pair of write function calls to tell it what you want to write (it may be possible to roll them into one, but all of the examples show it as two). Then you tell it you are done giving it the transaction data. Do not think it has actually sent the command though. After that, you tell it to send the command. Once that is done, you have to tell it to delete the transaction. In Arduino, this is all a single function call. In ESP-IDF, it is a disaster. I2C is a simple protocol! ESP-IDF manages to take something incredibly simple and turn it into something extremely complicated. This is perhaps the biggest reason I hate frameworks. This is not a one time thing either. Even Arduino's I2C driver is overly complicated to use. In the process of porting it, I eliminated a significant amount of setup cruft caused by using an object oriented approach where it is entirely unnecessary. In my experience, every framework has places where it takes something almost trivial and makes it more complicated than it has to be. Libraries sometimes do this too, for example, most high level languages built in libraries make pipes prohibitively hard to use (one of the few places Python really biffs it), but frameworks do this all over the place. Threading libraries tend to do this as well.
Compared to this, most of my other issues with frameworks are trivial. The worst one is that when a framework has a bug, it is a lot harder to fix than a bug in code you wrote yourself, even if the framework is well written, because you are not familiar with the framework's code. I have legitimately abandoned frameworks because they had one critical bug that would have taken me more time to fix than just writing the code myself, from scratch. Now days, I generally just skip the part where I use a framework in the first place. Another is that, in my experience, frameworks often take more time to use than writing code from scratch anyway, because learning a framework takes time. Unless I am going to use most of the features offered by the framework, it is not worth my time to learn it. I have written AJAX functions from scratch many times instead of using JQuery, because it is actually faster to do that than it is to setup JQuery on my web site and look up how to use its (admittedly high quality) AJAX stuff. Frameworks also consume unnecessary space, leading to memory bloat, performance costs, and unnecessary bandwidth costs, unless you are using almost 100% of the features provided. These things may seem trivial, but when you consider the scale on the server side instead of at the individual client, it adds up surprisingly fast. Frameworks also tend to come and go, especially in web development. This means that the framework you are learning today probably will not be worth much to you in a couple of years. It might help you advance in your current company, but most promotion in the software industry is not internal, and that other company you really want to work for probably does not use the framework you are learning for your current job. This is further aggravated by the fact that there are so many frameworks out there that there really is no point learning one that you do not have an immediate application for. Lastly, and this is really more important than the last few issues, frameworks are almost never as general purpose as libraries. Nearly all frameworks are designed with a fairly narrow set of use cases in mind, but then they are advertised with a focus on one design element that is not closely related to those use cases. For example, responsiveness is a popular feature of web frameworks that dynamically load data. Most such frameworks are designed for point-of-sale and inventory management, and they tend to be janky or even downright lousy for anything else. They do not tell you that you should not use them to make a site that needs good quality real-time interaction though. They do not tell you that the responsiveness comes at a heavy performance cost. So you decide to use a framework like AngularJS, because "responsive" sounds like exactly what you want for your browser game, but halfway through, after you have spent many hours learning the framework, you discover that while it might work well for business applications (if you are willing to tolerate the making-easy-things-excessively-complicated problem that it also has), it is a horrible choice for games, because what they mean by "responsive" is not the same as what you thought it meant. (No, I did not actually try to use AngularJS for a browser game. I did use it for an inventory management system though, where it worked fairly well but where the performance costs occasionally reared their ugly heads in ways that do not matter to an inventory management system but that would make a game unplayable.)
There are a lot of problems with frameworks. Their popularity makes it clear that a lot of people are willing to put up with those problems, for the perceived benefits, but in my experience, most of those benefits are ethereal. Recent research suggests that system and development architectures make a much bigger difference in success rates than development stacks anyway. This means that any benefits gained from using a particular framework will almost certainly be dwarfed by improving team communication and spending more time on quality design work. My personal experience supports this conclusion.
What it all comes down to is that if you really want to use a framework, pick one with features you are going to use, and know what you are getting yourself into. Personally, I find it easier and faster to code the features I want myself, and I find that the time spent finding and learning a suitable framework ultimately costs me more than just writing the code myself. You might say I am reinventing the wheel, but the fact is, when there are a million different kinds of track, you need a million different kinds of wheel if you want all of your trains to run smoothly.
Subscribe to:
Posts (Atom)