Narrative in video game design is much more important than in table top game design. In table top games, narrative is generally limited to providing a motive for the player to play the game. It explains why the player wants the most points, to control the most territory, or to complete some goal before everyone else. In video games, narrative is not a static answer to the questions, "Who am I?" and "Why am I here?" in the context of the game. In video games, the narrative is the story the player is either playing through or creating.
Some games do not need narratives. An excellent example of this is Tetris. There is neither room nor need for narrative in Tetris. Unlike table top games though, most video games are designed to be played for more than just an hour or two. Many games have transitions between levels, regions, or even activities, and each transition often needs some kind of narrative. Because the narratives for a particular game need to be associated with each other, that means games need stories, where each narrative is a progression of the previous narrative. While not all video games need narratives, most do, and many need narratives that fit together into a whole coherent story.
Video games are a form of art that turns many different artistic elements into complex compositions. The obvious artistic elements in video games are visual art, music, and sound effects. Stories are art too though, and many of the best games integrate narrative art forms into themselves. Unfortunately, narrative is frequently added to games as an afterthought. This can result in games with awesome graphics, audio, game mechanics, and so on, but with terrible, unmotivating stories. Narrative is often an essential element in the composition of video games, but it also tends to be overlooked, resulting in mediocre games.
Narrative in a game should answer a few basic questions. It should explain who the character is and why the character is there. This, in turn, should provide at least initial answers to the question of what the immediate goals of the character are, and ideally it should motivate the player to want to achieve those goals. In video games, the narrative should form a story, that explores the goals of the character, revealing new goals when necessary and helping the player understand how his or her actions are helping the character to achieve those goals.
Especially good narratives also explore the development of the character, and they can even ask hard questions designed to make the player think. For example, in a serious game, the narrative might prompt the player to consider whether the things the character is doing are wrong or right. It might ask questions that encourage the player to reconsider his or her point of view on something.
As with books, television shows, or movies, narratives can add spice by throwing in plot twists. Engaging the player with narrative is often just as important or even more important than engaging the player with pretty graphics, good music and sound effects, and well designed game mechanics.
Narrative also provides an additional setting for exploration, which can be a very strong motivator. Players like to explore, because they can find interesting things. This may be new scenery, interesting locations in the game, new items, new friends, or new enemies. A good narrative also provides room for exploration. The Myst series is an excellent example of this. There are many places where you can find books or other text that reveals back story, information about the relationships of the characters, how things work in the game worlds, and more. There are also occasional opportunities to explore narrative by talking to people in the game. A vast majority of the narrative, however, is revealed through settings found during physical exploration. In the original Myst, you can learn about the personalities of the brothers by exploring their dwellings in the various Ages that are available for you to explore. This provides more motivation for physical exploration. In Myst 3: Exile, a man that got caught up in the schemes of the brothers reveals back story and plot information at various points during the early and late game. This narrative exploration is at least as interesting as the physical exploration of the game. Wanting to know more of the story can be a very strong driving force, when the narrative is well designed.
For many genres of video game, narrative is critical. Not all games need narrative, but when a game needs narrative, it is important to do it well. Just like artists do not put epic paintings in crummy frames, because the frame will detract from the painting, game designers should not combine poor narratives with epic graphics, sound, and game mechanics, because the narrative will devalue all of the higher quality art work. The popularity of retro games has also shown that narrative quality is often more important to players than the quality of other art in a game. Good narrative might be hard to do, but it is very important to many kinds of video games.
Monday, April 16, 2018
Thursday, April 12, 2018
Video Game Development: Cameras
If you have not read Video Game Development: Relativity, you might want to do so, before reading this article. If you don't have a solid grasp of basic relativity, this article may be confusing and hard to follow.
While the concept of cameras is generally only a major topic in 3D graphics, this discussion is going to be limited to cameras looking at 2D game worlds. The reason for this is that cameras in 3D space is a whole, fairly complex topic of its own, and a full explanation would take many, fairly long articles. The purpose of this article is to explain how cameras work in 2D and why we need them.
One of the most important places for understanding relativity in video game development is in cameras. The most obvious reference frame for games is the screen or window, where the upper left corner is (0, 0). This works fine, until your game area is larger than the screen. What happens when your game is on an 800 by 600 screen, but the game map is several thousand pixels wide and tall? In most video games, a majority of the game map is off of the screen at any given time. To view the entire map, we have to be able to change where on the map the screen is looking. In my experience teaching video game design, the most obvious solution is to move the map around, keeping the screen as the reference frame. This is like the pre-Copernican heliocentric model of the solar system though. Every time the player wants to see a different area of the map, every single entity on the map has to be moved. This even includes entities that are not supposed to be able to move, like trees or buildings. As with the heliocentric model of the solar system, this results in a lot of extra math, and that can be very expensive when it comes to performance. The fact is, the screen makes a pretty poor reference frame, when the map is larger than the screen.
If there is a main character, it might be tempting to make the main character the reference frame. Unfortunately, this is no better than making the screen the reference frame. In fact, generally these are identical, as the screen is generally always centered on the character, and if the character is the reference frame, the position of the screen never moves. So again, we are back to the heliocentric analog, where we have to move everything, including stationary objects, whenever the character moves. As before, the character is a pretty poor reference frame as well.
What if we make the reference frame the thing that is supposed to be stationary? If everything is supposed to appear to be moving relative to the map, then maybe the map should be the reference frame. This allows stationary things to remain stationary, instead of wasting processing time to move with reference to the character or screen. But now we have a problem: How do we move everything relative to the screen, so we can see different parts of the map? This is the wrong question. This is like ancient astronomers asking, "How does everything move relative to the Earth?", and it is what resulted in the heliocentric model in the first place. The answer that lead to the truth was that we are moving, relative to something else that makes a better reference frame.
What if we allow the player to move? (It is important to distinguish here between character and player. The character is an entity in the game that is controlled by the player. The player is the person sitting at the desk in front of the computer.) The player's eyes, in the game world, are essentially the screen. So what I am suggesting is, what if the screen moves relative to the world? It turns out this is the right solution.
In a video game, the screen represents the player looking into the game world. It essentially represents a camera in the game world that is transmitting what it sees to the players screen. In this context, if we want the player to see a different part of the world, we just move the camera. In 2D games, the camera can be represented by a rectangle. Typically the rectangle will be the size of the screen (if it is not, that generally represents scaling), and the position of the rectangle in the game world determines what part of the world the player sees.
Implementation of a camera is actually fairly simple. The camera data structure contains four elements. It contains an x and y coordinate that together represent its position in the game world, and it contains a height and width, that represent the area in the game world that it can see. If that area is larger than the screen, that represents scaling down the image to fit that area onto the screen (essentially zooming out), and if that area is smaller than the screen, that represents scaling up the image to fill the screen area (zooming in). Typically, however, the camera size is either the same size as the screen, or it is the size of the portion of the screen where the game world will be displayed.
Once you have a rectangular representation of your camera, all that is necessary is a little bit of additional math when rendering. Consider, if an entity in the game is at (0, 0), and the camera is at (10, 10), then the entity will be rendered 10 pixels past the top and 10 pixels past the left of the camera. This position is (-10, -10). This suggests that all that is necessary to apply the camera is to subtract the camera from the position of each entity as it is being rendered. It turns out this will get the correct results. All you need to implement a camera is a representation of the camera as a rectangle and two subtractions for each entity being rendered.
Scaling requires a little bit more work. Typically, the camera is the same size as the area of the screen where the image will be rendered, but sometimes we want to make the camera smaller or larger, to scale the image. This allows the ability to zoom in or out. To scale, we have to divide the screen area by the camera area, and then scale the image being rendered by that value. For example, if our window is 100 by 100, and the camera is 200 by 200, then we need to scale the image by 0.5 by 0.5 (that is (100/200, 100/200)), for it to fit. Some rendering libraries will handle this automatically, if you tell them what size you want the source and destination to be. Others provide scaling functions that may be used by the developer, if desired. In 2D games, zooming is not a terribly common feature, but with a well implemented camera, it is not hard to accomplish.
Cameras come with some additional advantages. For example, if you perform collision detection of each game entity with the camera, you can avoid rendering things that are not on the screen, by only rendering entities that collide with the camera. This can result in significant performance improvements, as rendering is generally far more expensive than collision detections.
Another advantage of cameras is that the view is not bound to the map or any other game entity. For a game where the player controls a single character, keeping the character in the middle of the screen is as easy as centering the camera around the character each time the character moves. If you want the player to see some other part of the map for some reason though, it is easy to move the camera to see that location for as long as necessary. In games where the player should be able to view any part of the map as desired (real-time strategy, for example), the player can have complete control over the camera position. Adding this flexibility to a game that uses the screen or character as the reference frame is significantly more work than just moving a camera around. In addition, handling this with a camera is significantly more efficient.
Cameras are the simplest and most efficient way of providing a means for the player to view different parts of a game world. They are not initially obvious to everyone though. With a basic understanding of relativity, cameras are quite easy to implement, and they come with many benefits.
While the concept of cameras is generally only a major topic in 3D graphics, this discussion is going to be limited to cameras looking at 2D game worlds. The reason for this is that cameras in 3D space is a whole, fairly complex topic of its own, and a full explanation would take many, fairly long articles. The purpose of this article is to explain how cameras work in 2D and why we need them.
One of the most important places for understanding relativity in video game development is in cameras. The most obvious reference frame for games is the screen or window, where the upper left corner is (0, 0). This works fine, until your game area is larger than the screen. What happens when your game is on an 800 by 600 screen, but the game map is several thousand pixels wide and tall? In most video games, a majority of the game map is off of the screen at any given time. To view the entire map, we have to be able to change where on the map the screen is looking. In my experience teaching video game design, the most obvious solution is to move the map around, keeping the screen as the reference frame. This is like the pre-Copernican heliocentric model of the solar system though. Every time the player wants to see a different area of the map, every single entity on the map has to be moved. This even includes entities that are not supposed to be able to move, like trees or buildings. As with the heliocentric model of the solar system, this results in a lot of extra math, and that can be very expensive when it comes to performance. The fact is, the screen makes a pretty poor reference frame, when the map is larger than the screen.
If there is a main character, it might be tempting to make the main character the reference frame. Unfortunately, this is no better than making the screen the reference frame. In fact, generally these are identical, as the screen is generally always centered on the character, and if the character is the reference frame, the position of the screen never moves. So again, we are back to the heliocentric analog, where we have to move everything, including stationary objects, whenever the character moves. As before, the character is a pretty poor reference frame as well.
What if we make the reference frame the thing that is supposed to be stationary? If everything is supposed to appear to be moving relative to the map, then maybe the map should be the reference frame. This allows stationary things to remain stationary, instead of wasting processing time to move with reference to the character or screen. But now we have a problem: How do we move everything relative to the screen, so we can see different parts of the map? This is the wrong question. This is like ancient astronomers asking, "How does everything move relative to the Earth?", and it is what resulted in the heliocentric model in the first place. The answer that lead to the truth was that we are moving, relative to something else that makes a better reference frame.
What if we allow the player to move? (It is important to distinguish here between character and player. The character is an entity in the game that is controlled by the player. The player is the person sitting at the desk in front of the computer.) The player's eyes, in the game world, are essentially the screen. So what I am suggesting is, what if the screen moves relative to the world? It turns out this is the right solution.
In a video game, the screen represents the player looking into the game world. It essentially represents a camera in the game world that is transmitting what it sees to the players screen. In this context, if we want the player to see a different part of the world, we just move the camera. In 2D games, the camera can be represented by a rectangle. Typically the rectangle will be the size of the screen (if it is not, that generally represents scaling), and the position of the rectangle in the game world determines what part of the world the player sees.
Implementation of a camera is actually fairly simple. The camera data structure contains four elements. It contains an x and y coordinate that together represent its position in the game world, and it contains a height and width, that represent the area in the game world that it can see. If that area is larger than the screen, that represents scaling down the image to fit that area onto the screen (essentially zooming out), and if that area is smaller than the screen, that represents scaling up the image to fill the screen area (zooming in). Typically, however, the camera size is either the same size as the screen, or it is the size of the portion of the screen where the game world will be displayed.
Once you have a rectangular representation of your camera, all that is necessary is a little bit of additional math when rendering. Consider, if an entity in the game is at (0, 0), and the camera is at (10, 10), then the entity will be rendered 10 pixels past the top and 10 pixels past the left of the camera. This position is (-10, -10). This suggests that all that is necessary to apply the camera is to subtract the camera from the position of each entity as it is being rendered. It turns out this will get the correct results. All you need to implement a camera is a representation of the camera as a rectangle and two subtractions for each entity being rendered.
Scaling requires a little bit more work. Typically, the camera is the same size as the area of the screen where the image will be rendered, but sometimes we want to make the camera smaller or larger, to scale the image. This allows the ability to zoom in or out. To scale, we have to divide the screen area by the camera area, and then scale the image being rendered by that value. For example, if our window is 100 by 100, and the camera is 200 by 200, then we need to scale the image by 0.5 by 0.5 (that is (100/200, 100/200)), for it to fit. Some rendering libraries will handle this automatically, if you tell them what size you want the source and destination to be. Others provide scaling functions that may be used by the developer, if desired. In 2D games, zooming is not a terribly common feature, but with a well implemented camera, it is not hard to accomplish.
Cameras come with some additional advantages. For example, if you perform collision detection of each game entity with the camera, you can avoid rendering things that are not on the screen, by only rendering entities that collide with the camera. This can result in significant performance improvements, as rendering is generally far more expensive than collision detections.
Another advantage of cameras is that the view is not bound to the map or any other game entity. For a game where the player controls a single character, keeping the character in the middle of the screen is as easy as centering the camera around the character each time the character moves. If you want the player to see some other part of the map for some reason though, it is easy to move the camera to see that location for as long as necessary. In games where the player should be able to view any part of the map as desired (real-time strategy, for example), the player can have complete control over the camera position. Adding this flexibility to a game that uses the screen or character as the reference frame is significantly more work than just moving a camera around. In addition, handling this with a camera is significantly more efficient.
Cameras are the simplest and most efficient way of providing a means for the player to view different parts of a game world. They are not initially obvious to everyone though. With a basic understanding of relativity, cameras are quite easy to implement, and they come with many benefits.
Video Game Development: Relativity
Relativity is an important concept to understand in video game development. Yes, I am talking about Einstein's relativity. Don't worry, this will be fairly painless, because we don't need to get into the hard stuff. Relativity in general is something valuable to understand when developing video games. In some places it is even critical.
Relativity comes from the word "relative", and the initial assertion made by relativity is that everything on a material level is relative. More succinctly, there is no universal landmark. In a programming context, there is no detectable (0, 0) coordinate in real space. All spacial coordinate systems are relative to some reference frame. The reference frame is the landmark from which everything else is measured. In physics, this reference frame may be moving in space relative to something else, but because it is the reference frame, we treat it as stationary.
Copernicus and Galileo were instrumental in Western astronomy in the idea that the Sun is the center of the solar system, instead of the Earth being the center of the universe. But they were wrong. Yes, if the Sun is used as the reference frame, we can treat it as the center of the universe. But previous assumptions that everything orbited the Earth in strange heliocentric orbits were not strictly wrong. Using the Earth as a reference frame, they were actually just as correct as the Sun-centric models. The reason the Sun-centric model was superior was efficiency. For the solar system, the Earth centered model required a lot of excessively complex math to describe the orbits of other planets. The Sun centered model was far more efficient. But, when we look at bodies outside of our solar system, the Sun centered model is no better than the Earth centered model for things within our solar system. For this, the superior model is using the center of the galaxy as the reference frame. But, increasing scale is not inherently superior. A galaxy centered model for our solar system has the planets in complex helical orbits again. The ideal reference frame depends on what you are trying to measure, and it can be vastly different for different things.
This is what relativity is. It is the idea that there is no universal reference frame, and the ideal reference frame depends on what you are measuring. This idea is critical in video game development, and every game uses it, whether the developers realize it or not.
The most obvious reference frame for games is the screen or window, where the upper left corner is (0, 0). This fails, however, when game entities interact with each other. For example, when we do collision detection, we do not generally compare positions relative to (0, 0) to determine if there is a collision or not. It is possible, but the math is complex and hard to understand. Instead, we choose one object as a reference frame, and then we subtract its position from that of the second object. This subtraction is what makes the first object the reference frame, because the result of that subtraction is the position of the second object relative to the first. When we subtract like this, we shift the reference frame from the (0, 0) position on the screen or map, to the first object. From there, all we have to do is some simple size and position calculations to determine if there is a collision or not.
This extends beyond just collision detection. When we do any kind of physics or vector math to resolve interactions, we start by picking a reference frame object and subtracting it from any other object, to get the positions of each object with reference to the reference frame object. Again, it is possible in these interactions to use the screen or map as the reference frame, but it makes the math more complex and more difficult to understand. It can also make it less efficient. In some cases, using the wrong reference frame can serious impact game performance.
In video game development, relativity is essential. It is often used without even realizing it, but understanding how and why it works can provide a significant advantage in finding the best solution to difficult problems.
Relativity comes from the word "relative", and the initial assertion made by relativity is that everything on a material level is relative. More succinctly, there is no universal landmark. In a programming context, there is no detectable (0, 0) coordinate in real space. All spacial coordinate systems are relative to some reference frame. The reference frame is the landmark from which everything else is measured. In physics, this reference frame may be moving in space relative to something else, but because it is the reference frame, we treat it as stationary.
Copernicus and Galileo were instrumental in Western astronomy in the idea that the Sun is the center of the solar system, instead of the Earth being the center of the universe. But they were wrong. Yes, if the Sun is used as the reference frame, we can treat it as the center of the universe. But previous assumptions that everything orbited the Earth in strange heliocentric orbits were not strictly wrong. Using the Earth as a reference frame, they were actually just as correct as the Sun-centric models. The reason the Sun-centric model was superior was efficiency. For the solar system, the Earth centered model required a lot of excessively complex math to describe the orbits of other planets. The Sun centered model was far more efficient. But, when we look at bodies outside of our solar system, the Sun centered model is no better than the Earth centered model for things within our solar system. For this, the superior model is using the center of the galaxy as the reference frame. But, increasing scale is not inherently superior. A galaxy centered model for our solar system has the planets in complex helical orbits again. The ideal reference frame depends on what you are trying to measure, and it can be vastly different for different things.
This is what relativity is. It is the idea that there is no universal reference frame, and the ideal reference frame depends on what you are measuring. This idea is critical in video game development, and every game uses it, whether the developers realize it or not.
The most obvious reference frame for games is the screen or window, where the upper left corner is (0, 0). This fails, however, when game entities interact with each other. For example, when we do collision detection, we do not generally compare positions relative to (0, 0) to determine if there is a collision or not. It is possible, but the math is complex and hard to understand. Instead, we choose one object as a reference frame, and then we subtract its position from that of the second object. This subtraction is what makes the first object the reference frame, because the result of that subtraction is the position of the second object relative to the first. When we subtract like this, we shift the reference frame from the (0, 0) position on the screen or map, to the first object. From there, all we have to do is some simple size and position calculations to determine if there is a collision or not.
This extends beyond just collision detection. When we do any kind of physics or vector math to resolve interactions, we start by picking a reference frame object and subtracting it from any other object, to get the positions of each object with reference to the reference frame object. Again, it is possible in these interactions to use the screen or map as the reference frame, but it makes the math more complex and more difficult to understand. It can also make it less efficient. In some cases, using the wrong reference frame can serious impact game performance.
In video game development, relativity is essential. It is often used without even realizing it, but understanding how and why it works can provide a significant advantage in finding the best solution to difficult problems.
Monday, October 16, 2017
Ideological Programming
Today I was following what is actually a very good tutorial on SDL2, in C (despite it being written for C++). I came across an excessively complex section of code. The tutorial has me writing an initialization function. This is an excellent idea, as it groups a bunch of related actions together. Unfortunately, it is both hard to read and hard to write, and as far as I can tell, it is purely because the writer subscribes to a particular programming ideology.
This function starts by initializing a return variable, used to indicate failure or success. Then it initializes the video subsystem. If it fails, it sets the return variable to false, otherwise it goes into a nested conditional, creating a window and then testing for success, then it goes into another nested conditional for starting up the image loading subsystem, and only then does it get a surface to draw to. This is nested three levels deep. Each level tests if some initialization worked, and if not, it prints an error message and sets the return variable to false. The nested conditionals make the code hard to read and hard to write. There is a reason the writer of the tutorial did this though. It is an ideological reason.
There is this programming ideology that says a function should only have a single return point. In other words, it should not return anywhere except at the end. Programmers who subscribe to this tend to use return variables and nested conditionals to avoid returning in a conditional before the end. What is the benefit of this ideology? There is none. It is based on the idea of mathematical functions. Mathematical functions can only have a single return point, and some programmers believe that this should be applied to functions in programming. The fact, however, is that the word "function" means very different things to mathematics and programming. Programming functions are loosely based on mathematical functions, but they are not, by any stretch, the same. There is one exception: In functional programming languages, functions are analogous to mathematical functions. As such, pure functional languages don't allow multiple return points in functions (this may not always be obvious though). C is not a functional language though. There is no benefit to adding this artificial programming constraint. In fact, in the worse case, it will result in inferior machine code. While a well designed compiler might be able to optimize away most of the extra work for nested conditionals to get to the return point when a failure case occurs, it will never end up better than a set of conditionals where a failure just returns inside the conditional testing for failure, instead of using a return variable and nesting. In many cases it will end up worse though. Intermediate return points avoid unnecessary work, especially in a case like this. More importantly though, the flattened version of the code is both more readable and easier to comment. The ideology might sound good, but it is applying pure math principles to a more complex and flexible discipline, and in doing so it produces worse outcomes.
This is not the only programming ideology that is expensive and wasteful without producing any benefit. Object orientation as a programming paradigm is founded on the ideology that everything should be an object. Following this ideology results in hard to read code that uses significantly more memory than necessary and performs poorly on modern computing systems. Programming to a particular ideology might have some academic value, but in real life situations, it generally just causes problems.
When writing a program, these should be the primary goals, starting with the most important:
Ideological programming can be fun as a hobby or a purely academic pursuit, but in the real world, following abstract rules that have no practical value is a waste of everyone's time and money. Effective programming is about using your tools efficiently and effectively. When you eliminate a tool from your toolbox for a purely ideological reason, you artificially limit your skill and the quality of your work. Just say no to ideological programming.
This function starts by initializing a return variable, used to indicate failure or success. Then it initializes the video subsystem. If it fails, it sets the return variable to false, otherwise it goes into a nested conditional, creating a window and then testing for success, then it goes into another nested conditional for starting up the image loading subsystem, and only then does it get a surface to draw to. This is nested three levels deep. Each level tests if some initialization worked, and if not, it prints an error message and sets the return variable to false. The nested conditionals make the code hard to read and hard to write. There is a reason the writer of the tutorial did this though. It is an ideological reason.
There is this programming ideology that says a function should only have a single return point. In other words, it should not return anywhere except at the end. Programmers who subscribe to this tend to use return variables and nested conditionals to avoid returning in a conditional before the end. What is the benefit of this ideology? There is none. It is based on the idea of mathematical functions. Mathematical functions can only have a single return point, and some programmers believe that this should be applied to functions in programming. The fact, however, is that the word "function" means very different things to mathematics and programming. Programming functions are loosely based on mathematical functions, but they are not, by any stretch, the same. There is one exception: In functional programming languages, functions are analogous to mathematical functions. As such, pure functional languages don't allow multiple return points in functions (this may not always be obvious though). C is not a functional language though. There is no benefit to adding this artificial programming constraint. In fact, in the worse case, it will result in inferior machine code. While a well designed compiler might be able to optimize away most of the extra work for nested conditionals to get to the return point when a failure case occurs, it will never end up better than a set of conditionals where a failure just returns inside the conditional testing for failure, instead of using a return variable and nesting. In many cases it will end up worse though. Intermediate return points avoid unnecessary work, especially in a case like this. More importantly though, the flattened version of the code is both more readable and easier to comment. The ideology might sound good, but it is applying pure math principles to a more complex and flexible discipline, and in doing so it produces worse outcomes.
This is not the only programming ideology that is expensive and wasteful without producing any benefit. Object orientation as a programming paradigm is founded on the ideology that everything should be an object. Following this ideology results in hard to read code that uses significantly more memory than necessary and performs poorly on modern computing systems. Programming to a particular ideology might have some academic value, but in real life situations, it generally just causes problems.
When writing a program, these should be the primary goals, starting with the most important:
- The project is completed within acceptable margins for cost and schedule.
- The program produces the correct output for every possible input.
- The program is cheap and easy to maintain.
Ideological programming can be fun as a hobby or a purely academic pursuit, but in the real world, following abstract rules that have no practical value is a waste of everyone's time and money. Effective programming is about using your tools efficiently and effectively. When you eliminate a tool from your toolbox for a purely ideological reason, you artificially limit your skill and the quality of your work. Just say no to ideological programming.
Monday, June 19, 2017
Why Are Pipes Hard?
I learned something amazing today. I have done a few small multi-processing tasks in the past, some using Python and one using Java. The most appropriate form of interprocess communication for these was pipes. The experience was always very painful. I had to figure out how to cobble together the right objects to produce a usable communication system. I had to learn some special uses of objects to allow the parent process to continue executing, instead of waiting for the child to finish. Working out the bugs was very difficult, because it was hard to tell if the problem was my communication protocol, if I had connected my objects wrong, or if maybe I had given an advanced function or method some incorrect parameter. Of course, I expect this sort of difficulty. I had heard from others just how hard interprocess communication is, especially with pipes. That is just the way it is, I guess.
Today, a friend sent me a link to this. That document discusses several types of interprocess communication. There is one chapter on pipes, and the author starts the chapter, "There is no form of IPC that is simpler than pipes." I found this rather amusing and slightly confusing, as this was not my experience. Shortly following that, he adds that the document will not spend much time on pipes because they are so easy. This was confusing. By the end of the section, I knew two important things. First, the author is right; pipes are incredibly easy. Second, everyone else is doing pipes wrong.
In Python, you have to create and pass pipes into the function call that starts a new process. I forget the exact details, but I seem to recall having to use a different function or at least put some obscure parameter into the regular one to get it to return before the child process was completed. Then I had to use some method of the pipe object to interact with the child, and this was rather difficult to get right. In one case I did have some issues with my communication protocol, but I thought it was yet another issue with how I was trying to use the pipe. It took half an hour to figure out that the problem was some trivial thing with how was formatting the data I was passing through the pipe. In short, the system was completely opaque and very difficult to use. Java was not much better (in fact, I think I gave up on communication while the child process was still running, and just made the program call multiple times). The friend who sent me the link told me he had the same experience when he read that section, except with the Go programming language.
Now, that document describes how to use pipes in C. First, you create a pipe. This requires merely making a single function call, passing in a two integer wide array. The array is populated with a pair of file handles for the reading and writing end of the pipe respectively. Then, you treat those file handles like you would any regular file! The main difference is that they are more like standard in and standard out than a file on disk, but any programmer worth his or her salt should know how to deal with that! The document goes on to use read and write calls to send and receive data, but you could easily use any function that reads or writes files for this. If there is anything complicated about pipes, it is only that they have limited space, but 10kb is a pretty big limit. Once you have a pipe, you just fork the process, which will provide the child process with copies of the file descriptors to the pipe. Now the two processes can communicate with read and write calls (or other file I/O calls) on the pipe. Of course, if you need two way communication, you will need two pipes, but that is trivial.
The fact is, pipes are incredibly easy! For some reason though, only low level languages can manage to get them right. I don't often have something bad to say about Python, but this is one place where Python got it totally wrong! I have plenty of bad things to say about Java, so it does not really surprise me that Java missed the mark on this one. So why are pipes so hard in language that are higher level than C or C++? In fact, I am almost certain that pipes are easier in assembly language than in higher level languages (and I know what I am talking about here). Clearly the operating system has it right. Why can't high level languages get it right? They are supposed to make programming easier if I am not mistaken, but when I start considering writing a C module for Python, because that would be so much easier than just using the built in, you know something is wrong.
The problem is not terribly difficult for me to identify, but much of the programming world is so in love with Object Oriented Programming that they don't want hear anything bad about it, let alone admit that it might have a serious problem. The problem here is that the OS gets it right in the first place, so it does not need complicated wrappers to make it easier. At the OS level, pipes are about as simple as it gets. C is wise to just leave it alone and let it do its thing. Python, Java, and other languages, however, feel obligated to wrap everything of objects, because hey, object oriented. The fact is pipes don't need objects. They are perfectly fine just as they are. Wrapping them in objects makes them absurdly more complicated and difficult to use, without adding anything of value. I was actually stunned when I saw the C code in that document. I had a hard time believing that it was correct, because my experience did not support that conclusion. Using objects where they are unnecessary is bad design, and we really need to teach programmers to stop doing it. There are good reasons to use objects, but there are also good reasons not to.
I have talked about this before, but the problem here is OOP, and that is just as absurd as Integer Oriented Programming (or Bit Oriented Programming, as a friend suggested). When we decide to orient all of our programming around a single data structure (or metadata structure, in this case), we limit ourselves. We destroy our ability to make good decisions about what tools will be the best for each problem. Objects are a great tool, but we don't use a chain saw to screw in a screw or hammer in a nail. Honestly, I feel like multi-processing in Python or Java is just like following some really complicated instructions for how to push a peg into a hole with a table saw without ever touching the peg or injuring yourself. It would be really nice if those languages would just let me pick up the peg and push it into the hole with my finger, instead of trying to build a huge complex machine around such a simple process.
Today, a friend sent me a link to this. That document discusses several types of interprocess communication. There is one chapter on pipes, and the author starts the chapter, "There is no form of IPC that is simpler than pipes." I found this rather amusing and slightly confusing, as this was not my experience. Shortly following that, he adds that the document will not spend much time on pipes because they are so easy. This was confusing. By the end of the section, I knew two important things. First, the author is right; pipes are incredibly easy. Second, everyone else is doing pipes wrong.
In Python, you have to create and pass pipes into the function call that starts a new process. I forget the exact details, but I seem to recall having to use a different function or at least put some obscure parameter into the regular one to get it to return before the child process was completed. Then I had to use some method of the pipe object to interact with the child, and this was rather difficult to get right. In one case I did have some issues with my communication protocol, but I thought it was yet another issue with how I was trying to use the pipe. It took half an hour to figure out that the problem was some trivial thing with how was formatting the data I was passing through the pipe. In short, the system was completely opaque and very difficult to use. Java was not much better (in fact, I think I gave up on communication while the child process was still running, and just made the program call multiple times). The friend who sent me the link told me he had the same experience when he read that section, except with the Go programming language.
Now, that document describes how to use pipes in C. First, you create a pipe. This requires merely making a single function call, passing in a two integer wide array. The array is populated with a pair of file handles for the reading and writing end of the pipe respectively. Then, you treat those file handles like you would any regular file! The main difference is that they are more like standard in and standard out than a file on disk, but any programmer worth his or her salt should know how to deal with that! The document goes on to use read and write calls to send and receive data, but you could easily use any function that reads or writes files for this. If there is anything complicated about pipes, it is only that they have limited space, but 10kb is a pretty big limit. Once you have a pipe, you just fork the process, which will provide the child process with copies of the file descriptors to the pipe. Now the two processes can communicate with read and write calls (or other file I/O calls) on the pipe. Of course, if you need two way communication, you will need two pipes, but that is trivial.
The fact is, pipes are incredibly easy! For some reason though, only low level languages can manage to get them right. I don't often have something bad to say about Python, but this is one place where Python got it totally wrong! I have plenty of bad things to say about Java, so it does not really surprise me that Java missed the mark on this one. So why are pipes so hard in language that are higher level than C or C++? In fact, I am almost certain that pipes are easier in assembly language than in higher level languages (and I know what I am talking about here). Clearly the operating system has it right. Why can't high level languages get it right? They are supposed to make programming easier if I am not mistaken, but when I start considering writing a C module for Python, because that would be so much easier than just using the built in, you know something is wrong.
The problem is not terribly difficult for me to identify, but much of the programming world is so in love with Object Oriented Programming that they don't want hear anything bad about it, let alone admit that it might have a serious problem. The problem here is that the OS gets it right in the first place, so it does not need complicated wrappers to make it easier. At the OS level, pipes are about as simple as it gets. C is wise to just leave it alone and let it do its thing. Python, Java, and other languages, however, feel obligated to wrap everything of objects, because hey, object oriented. The fact is pipes don't need objects. They are perfectly fine just as they are. Wrapping them in objects makes them absurdly more complicated and difficult to use, without adding anything of value. I was actually stunned when I saw the C code in that document. I had a hard time believing that it was correct, because my experience did not support that conclusion. Using objects where they are unnecessary is bad design, and we really need to teach programmers to stop doing it. There are good reasons to use objects, but there are also good reasons not to.
I have talked about this before, but the problem here is OOP, and that is just as absurd as Integer Oriented Programming (or Bit Oriented Programming, as a friend suggested). When we decide to orient all of our programming around a single data structure (or metadata structure, in this case), we limit ourselves. We destroy our ability to make good decisions about what tools will be the best for each problem. Objects are a great tool, but we don't use a chain saw to screw in a screw or hammer in a nail. Honestly, I feel like multi-processing in Python or Java is just like following some really complicated instructions for how to push a peg into a hole with a table saw without ever touching the peg or injuring yourself. It would be really nice if those languages would just let me pick up the peg and push it into the hole with my finger, instead of trying to build a huge complex machine around such a simple process.
Thursday, June 1, 2017
Reusable Software Design
I just wrote about reusable software, and I figured reusable software design deserves its own article. Perhaps the biggest problem I identified with reusable code is that if it is has problems, the time required to learn the code just to fix it largely negates any value in using it in the first place. If we could easily understand reusable code enough to consistently adapt it to our own use cases, it would be perfect, but that is not how code works. Code is inherently hard to understand. Often, we even have a hard time understanding our own code after a few weeks of separation. Isn't there some way we can consistently benefit from existing work, without the risk of having to spend so much time learning how it works that the cost is greater than the benefits? It turns out that there is, and in some ways it may even be able to mitigate the costs of choosing not to use reusable code. The solution is reusable design!
Design patterns have formally existed since object oriented programming became a thing. Most design patterns are built around OOP. They don't have to be though. Things like loops and coherent if statements are primitive design patterns. Functions are a kind of design pattern as well. We often don't recognize this, because these have become integral elements of programming, but it is true. These are all examples of design patterns that don't rely on object orientation. In fact, very few design patterns require OOP. Around a year ago, I wrote an event handler that uses the Event Queue design pattern, in C, and I have used the Observer pattern in C as well. Any programming style or language can use design patterns (even assembly language). Design patterns are not limited to OOP languages.
Design patterns are superior to concrete code for reuseability, because they are more abstract and flexible. Design patterns don't specify implementation. They merely provide a template. For example, the "for loop" design pattern does not specify the exact number of iterations, the data type of the index variable, or the iteration step. We can use it to do simple counting, to step through a string one (or two, or five...) character at a time, or to traverse an array. These are flexible implementation details. The if statement may or may not have an else or if-else clause. The expression used to generate the boolean value used as the conditional is not defined for us. We can use if statements for all kinds of things, because implementation is not defined for us. Design patterns are flexible enough that it is trivial to adapt them to a particular application.
Good design can save a lot of coding time. A friend of mine likes to say, "Hours of coding can save minutes of design." Reusable code is generally endorsed for how much time it can save developers, but it comes with potentially serious costs. Good design may take longer than crummy design, but it always pays for itself many times over. Good design can save more time than using reusable code can. Reusable design (design patterns) can save significant amounts of design time. This is a double win, because judicious use of reusable design comes with no additional costs, and time spent (or saved) on design is generally more valuable than time spent on implementation. When we save design time, we also save significantly more coding time.
Good design can reduce the need for reusable code, by finding ways to make coding from scratch faster than learning and using an external library or framework. When reusable code is the best option, good design can optimize its use to reduce the costs.
Reusable design is, in my opinion, of far greater value than reusable code. Design patterns are generic enough to solve a large number of problems, and they are easy enough to understand to customize them to a particular application. Because design time is more valuable than coding time, they also have the potential to save far more time than reusable code. Reusable design is probably something we should spend significantly more time and effort on.
Design patterns have formally existed since object oriented programming became a thing. Most design patterns are built around OOP. They don't have to be though. Things like loops and coherent if statements are primitive design patterns. Functions are a kind of design pattern as well. We often don't recognize this, because these have become integral elements of programming, but it is true. These are all examples of design patterns that don't rely on object orientation. In fact, very few design patterns require OOP. Around a year ago, I wrote an event handler that uses the Event Queue design pattern, in C, and I have used the Observer pattern in C as well. Any programming style or language can use design patterns (even assembly language). Design patterns are not limited to OOP languages.
Design patterns are superior to concrete code for reuseability, because they are more abstract and flexible. Design patterns don't specify implementation. They merely provide a template. For example, the "for loop" design pattern does not specify the exact number of iterations, the data type of the index variable, or the iteration step. We can use it to do simple counting, to step through a string one (or two, or five...) character at a time, or to traverse an array. These are flexible implementation details. The if statement may or may not have an else or if-else clause. The expression used to generate the boolean value used as the conditional is not defined for us. We can use if statements for all kinds of things, because implementation is not defined for us. Design patterns are flexible enough that it is trivial to adapt them to a particular application.
Good design can save a lot of coding time. A friend of mine likes to say, "Hours of coding can save minutes of design." Reusable code is generally endorsed for how much time it can save developers, but it comes with potentially serious costs. Good design may take longer than crummy design, but it always pays for itself many times over. Good design can save more time than using reusable code can. Reusable design (design patterns) can save significant amounts of design time. This is a double win, because judicious use of reusable design comes with no additional costs, and time spent (or saved) on design is generally more valuable than time spent on implementation. When we save design time, we also save significantly more coding time.
Good design can reduce the need for reusable code, by finding ways to make coding from scratch faster than learning and using an external library or framework. When reusable code is the best option, good design can optimize its use to reduce the costs.
Reusable design is, in my opinion, of far greater value than reusable code. Design patterns are generic enough to solve a large number of problems, and they are easy enough to understand to customize them to a particular application. Because design time is more valuable than coding time, they also have the potential to save far more time than reusable code. Reusable design is probably something we should spend significantly more time and effort on.
Reusable Software
Ever since the invention of modular programming, reusable software has been a huge deal. Nearly ever programming language comes with build-in reusable parts or some kind of standard library. A great number of articles have been written, extolling the many virtues of reusable software components. Enormous repositories of reusable parts have been created. Most problem spaces have complex ecosystems of frameworks, libraries, and sometimes even just short snippets of code or individual reusable functions. Anything this good has to come at a cost though, and the costs of reusable software are almost universally ignored.
First I want to say that reusable software is awesome. I am not trying to bash it here. Reusable software has many great benefits. It saves a ton of programming time. Popular reusable components tend to be fairly high quality, because they have already been tried, tested, fixed, and improved. Reusable components are typically fairly easy to learn and use, because no one wants to use something that is more expensive to learn than writing it fresh would be, and no one wants to use something that takes more effort to use than it would take to roll fresh. Reusable software can be used to make reasonably high quality software far faster than writing it from scratch most of the time.
Reusable software is a two edged sword, and most programmers cannot see the edge that is facing them. Reusable software is so good that it is often difficult to see the costs, and even when the costs are known, they are often ignored. Most of the time, the costs of reusable software are acceptable, but occasionally they can cause serious problems. For example, recently a large number of web sites stopped working because a trivial piece of reusable code was removed from a popular nodeJS repository. The time savings for using the code was probably between 1 and 5 minutes for a single developer. The total cost in web site down time was probably in the hundreds of thousands or millions of dollars. This is not a common scenario, but it is one that could have easily be avoided by spending a few minutes to write a trivial piece of code, instead of relying on an external source to remain available forever.
Perhaps the most obvious cost of reusable software is fitness of a particular purpose. To be useful, reusable code must be usable for a variety of applications. This means it must be generic. On cost of making something generic is that it becomes less suitable for nearly every application, especially very specialized ones. For flexible applications that are not critical, the application can generally be adapted to the reusable code. This will almost certainly affect design, but it rare affects the utility and usability of the application. Sometimes, however, it does. Some specialized applications have strict design requirements that a generic component would violate. In these cases, reusable software just cannot be used. Occasionally, reusable software will seriously limit the design choices of even more mundane software in ways that are unacceptable. In these cases, reusable software should not be used. If using a particular framework or library feels like trying to fit a square peg in a round hole, it may be time to consider alternatives, including writing the code from scratch.
Another fairly obvious but deliberately ignored cost of reusable software is performance. Making something generic means making it suitable for a wide variety of use cases. This means a lot of features and a lot of "just in case" elements. These use memory and processing power, whether you actually need the features or not. In many applications, this does not make a huge difference. In some though, it can make an enormous difference. In applications that are performance critical, reusable code in bottlenecks can be a major problem. Even in applications where performance is not critical, bloated or slow reusable code can be a problem. Modern computers never run just one application at a time. The typical computer is running 10 to 50 processes at any given moment, and all of those processes have to share resources. While it might seem fine to waste a few megabytes here and there, when there are 50 processes wasting a few megabytes each, memory use can become a serious problem. Similarly, processor time also must be shared between processes, and a process that is wasteful can affect the performance of the entire machine. For many kinds of applications, this is rarely a problem but for some (web pages, for example, where a user may have anywhere from 5 to 100 tabs open at a time), a little resource hogging can go a long way. In general, it is a good idea to keep in mind that reusable code is consuming resources for all of its features, even those you are not using. A good rule of thumb is that if you are not using more than one or two features of a framework, it might be time to consider looking for something lighter or writing the code from scratch.
Reusable software makes code less maintainable. This is very counter-intuitive. There is a general assumption that using a framework or library takes some of the maintenance burden off of the programmer, and this is true. Reusable code can definitely reduce the burden of maintenance, but this is not about maintenance time spent or saved. This is about being able to maintain the code in the first place. Learning exactly how a piece of reusable code works takes enough time that it defeats the purpose of using it, so few programmers bother. If the reusable code itself has a bug, however, the only options are either to ditch the reusable code and home brew a replacement, pay to have someone spend hours, days, or weeks learning how the code works and fixing the bug, or report the bug to the project and wait for it to get fixed upstream. The second option is so expensive that it is hardly ever an option. The typical solution is to sacrifice quality, utility, and usability by working around the bug. In addition to this, because reusable code is significantly harder to change (because the programmers using it are not intimately familiar with its code), valuable design changes to a program may be impossible. As with bugs, if a valuable design change is needed that does not work with the reusable software, the options are to ditch it and home brew a replacement, or pay someone for a lot of labor to figure out how to adapt the reusable code. Of course, once the reusable code has been altered, updates from the original source become invalid, and the reusable code must be maintained entirely in-house, at additional expense. Again, this is not necessarily a problem that will come up a lot. Major design changes after delivery are not exactly common. Reusable code does tend to be more well tested as well, so bugs in it will be more rare. When they do come up though, they can be very expensive.
Overall, reusable software is very valuable, and nearly ever program uses it in some way. It also comes with some costs though, and understanding those costs is important to get the most out of it. Sometimes it is better to write the code from scratch. Code written to the specific application will always be superior to reusable code. Reusable code can ultimately be more expensive than coding from scratch. Cases where reusable code costs more than writing code from scratch are generally uncommon, but determining this should be part of the cost analysis of any project. In addition, there are different levels of reusable software that should be considered. Using something like the C standard library or other build-in functions and features of a language is generally a given. For any popular language, these have been tested and optimized more than anything else. Libraries for hardware interaction are often necessary as well, though there may be multiple options. These tend to have more testing behind them as well. Convenience and aesthetics libraries (jQuery, for example) are typically far less critical, so they tend to have less rigorous testing, and the benefits they provide are less critical. Less essential libraries that have been around for less time deserve the most scrutiny. Very small pieces of reusable code also deserve scrutiny, because it may end up taking more time and resources in the long run than just writing it from scratch. Despite its high value, reusable code is not a silver bullet. It comes with its own costs, and sometimes those costs can be much greater than any benefits. Our industry needs to spend a bit more time considering the options and potential consequences than it currently does, and if it did, things would work a lot more smoothly for everyone.
First I want to say that reusable software is awesome. I am not trying to bash it here. Reusable software has many great benefits. It saves a ton of programming time. Popular reusable components tend to be fairly high quality, because they have already been tried, tested, fixed, and improved. Reusable components are typically fairly easy to learn and use, because no one wants to use something that is more expensive to learn than writing it fresh would be, and no one wants to use something that takes more effort to use than it would take to roll fresh. Reusable software can be used to make reasonably high quality software far faster than writing it from scratch most of the time.
Reusable software is a two edged sword, and most programmers cannot see the edge that is facing them. Reusable software is so good that it is often difficult to see the costs, and even when the costs are known, they are often ignored. Most of the time, the costs of reusable software are acceptable, but occasionally they can cause serious problems. For example, recently a large number of web sites stopped working because a trivial piece of reusable code was removed from a popular nodeJS repository. The time savings for using the code was probably between 1 and 5 minutes for a single developer. The total cost in web site down time was probably in the hundreds of thousands or millions of dollars. This is not a common scenario, but it is one that could have easily be avoided by spending a few minutes to write a trivial piece of code, instead of relying on an external source to remain available forever.
Perhaps the most obvious cost of reusable software is fitness of a particular purpose. To be useful, reusable code must be usable for a variety of applications. This means it must be generic. On cost of making something generic is that it becomes less suitable for nearly every application, especially very specialized ones. For flexible applications that are not critical, the application can generally be adapted to the reusable code. This will almost certainly affect design, but it rare affects the utility and usability of the application. Sometimes, however, it does. Some specialized applications have strict design requirements that a generic component would violate. In these cases, reusable software just cannot be used. Occasionally, reusable software will seriously limit the design choices of even more mundane software in ways that are unacceptable. In these cases, reusable software should not be used. If using a particular framework or library feels like trying to fit a square peg in a round hole, it may be time to consider alternatives, including writing the code from scratch.
Another fairly obvious but deliberately ignored cost of reusable software is performance. Making something generic means making it suitable for a wide variety of use cases. This means a lot of features and a lot of "just in case" elements. These use memory and processing power, whether you actually need the features or not. In many applications, this does not make a huge difference. In some though, it can make an enormous difference. In applications that are performance critical, reusable code in bottlenecks can be a major problem. Even in applications where performance is not critical, bloated or slow reusable code can be a problem. Modern computers never run just one application at a time. The typical computer is running 10 to 50 processes at any given moment, and all of those processes have to share resources. While it might seem fine to waste a few megabytes here and there, when there are 50 processes wasting a few megabytes each, memory use can become a serious problem. Similarly, processor time also must be shared between processes, and a process that is wasteful can affect the performance of the entire machine. For many kinds of applications, this is rarely a problem but for some (web pages, for example, where a user may have anywhere from 5 to 100 tabs open at a time), a little resource hogging can go a long way. In general, it is a good idea to keep in mind that reusable code is consuming resources for all of its features, even those you are not using. A good rule of thumb is that if you are not using more than one or two features of a framework, it might be time to consider looking for something lighter or writing the code from scratch.
Reusable software makes code less maintainable. This is very counter-intuitive. There is a general assumption that using a framework or library takes some of the maintenance burden off of the programmer, and this is true. Reusable code can definitely reduce the burden of maintenance, but this is not about maintenance time spent or saved. This is about being able to maintain the code in the first place. Learning exactly how a piece of reusable code works takes enough time that it defeats the purpose of using it, so few programmers bother. If the reusable code itself has a bug, however, the only options are either to ditch the reusable code and home brew a replacement, pay to have someone spend hours, days, or weeks learning how the code works and fixing the bug, or report the bug to the project and wait for it to get fixed upstream. The second option is so expensive that it is hardly ever an option. The typical solution is to sacrifice quality, utility, and usability by working around the bug. In addition to this, because reusable code is significantly harder to change (because the programmers using it are not intimately familiar with its code), valuable design changes to a program may be impossible. As with bugs, if a valuable design change is needed that does not work with the reusable software, the options are to ditch it and home brew a replacement, or pay someone for a lot of labor to figure out how to adapt the reusable code. Of course, once the reusable code has been altered, updates from the original source become invalid, and the reusable code must be maintained entirely in-house, at additional expense. Again, this is not necessarily a problem that will come up a lot. Major design changes after delivery are not exactly common. Reusable code does tend to be more well tested as well, so bugs in it will be more rare. When they do come up though, they can be very expensive.
Overall, reusable software is very valuable, and nearly ever program uses it in some way. It also comes with some costs though, and understanding those costs is important to get the most out of it. Sometimes it is better to write the code from scratch. Code written to the specific application will always be superior to reusable code. Reusable code can ultimately be more expensive than coding from scratch. Cases where reusable code costs more than writing code from scratch are generally uncommon, but determining this should be part of the cost analysis of any project. In addition, there are different levels of reusable software that should be considered. Using something like the C standard library or other build-in functions and features of a language is generally a given. For any popular language, these have been tested and optimized more than anything else. Libraries for hardware interaction are often necessary as well, though there may be multiple options. These tend to have more testing behind them as well. Convenience and aesthetics libraries (jQuery, for example) are typically far less critical, so they tend to have less rigorous testing, and the benefits they provide are less critical. Less essential libraries that have been around for less time deserve the most scrutiny. Very small pieces of reusable code also deserve scrutiny, because it may end up taking more time and resources in the long run than just writing it from scratch. Despite its high value, reusable code is not a silver bullet. It comes with its own costs, and sometimes those costs can be much greater than any benefits. Our industry needs to spend a bit more time considering the options and potential consequences than it currently does, and if it did, things would work a lot more smoothly for everyone.
Subscribe to:
Posts (Atom)