Hi Rolf, As far as my being "emotional", I'm just getting tired of crappy software. It happens after almost two decades in tech support, I suppose. Looking at leaving the industry in a few years -- just because it never really gets any better. I could go manage a support team, but I've been-there, done-that, and would only enjoy it in certain sized organizations. By the time the bugs are worked out of something, it's scrapped and a whole new thing full of bugs gets shipped to the customers. Not picking on my current employer -- this has been true at ALL of them. Software is seen as something that SHOULD be replaced regularly, like air filters -- not critical infrastructure like highways and bridges to get somewhere. This is sad. The bugs get old, since you THINK they hire top-notch engineers who have plenty of experience and know better, but then in every product there's stupid programming mistakes, pointers falling off the end of stacks, memory allocation screw ups, the usual. Very few of the bugs I've found over the years are "interesting", they're all the same old programmer screw-ups, every year. Different developers, all good people, release the same bugs. You'd think someone would notice and find a way to stop those. As I read more and more of the BS from the software industry giving lip-service to actually trying to find ways to stop these things from happening over and over again, they never come to pass. Companies find the solution is "too expensive", "too time consuming", or worse, you get those responses like I got here in the list yesterday from the engineer, "Boy, programming like that just wouldn't be any fun!" My favorite false-premise in software development is that of "code re-use". To be honest, PICsters seem to actually do it more than most. But in companies? Nah... you're paid to code... so you code... even if there was already code to do X checked into the CVS tree somewhere. Let me whip out the world's smallest violin for that guy who said highly disciplined coding wouldn't be any fun, and play it. It's not SUPPOSED to be fun, it's supposed to be a PROFESSION, and the software is supposed to get BETTER over time. Not be the same old re-hashed errors. Would it be better if the software releases just had new features, and weren't also bug-fix "service packs"? Yep. But no one cares anymore... customers, software vendors, doesn't matter... everyone's just "used to" it all sucking most of the time. Developers rant and rave about the 80/20 rule, but in reality, people want 100% of the things in their software to WORK. There's no "80" that use all the same features. Ask any tech support department if they want the 20% with bugs, fixed... and how much pain and overtime and late nights that 20% causes. The software industry is just so full of excuses these days, it's sad. Imagine if you bought a car and it behaved as badly as most software does. Or a house. You'd go looking for a better product. "I'm sorry sir, but you're attempting to use the 20% of your car/house/whatever that most people don't use. I'll turn in a trouble ticket and if enough people attempt to use that portion of the product, we might not "de-scope" the bug report out of the next version in lieu of new features, since new features bring in more revenue than fixing your problem." Getting tired of saying that, but in politically correct ways. Really tired. You bought something from us, but you're calling me telling me it doesn't work, and I have no ability to fix it for you. This is PARTICULARLY bad on one particular product I currently work on, because it's being phased out. Development of ANYTHING on it is literally decided on a case by case, ticket by ticket, basis -- and "end of sale" has already been announced. It had potential to be rock-solid, but Engineering/Product Management is moving on to other things. There's no direct replacement in the new product lines. It's extra painful for the support team. So I apologize if I got emotional and challenged you a bit. "Software never works right the first time" is so ingrained in the culture now, that it'll be virtually impossible to eradicate. The only good news is? It means both you and I have permanent jobs, so to speak... If the engineers at any company could put the tech support people at that company out of business, they'd have accomplished something. So far, no company I've worked for yet hasn't shown quarter over quarter growth in people paying for service contracts. I immensely enjoy reading http://joelonsoftware.com -- that guy seems to "get it", but how many companies do? I can read that site for hours, and have tried his software, and it *is* measurably better than most. I also love his discussion of how his tech support people are hired on a MANAGEMENT track, and required (at the company's expense) to go get a Master's degree in technology management. (And the perks for his developers are amazing, too -- but I'm not a developer.) The guy knows how to create his own culture and success, and does it. His company does all that and still turns a profit making software. Pretty impressive. He speaks highly of Apple, as do I, too. Their stuff "just works" most of the time, for me anyway... His article on shutting down a Windows machine: http://www.joelonsoftware.com/items/2006/11/21.html Contrasted with the fact that I just close the lid on my MacBook, and it does what it's supposed to. If I need it "All the way off" I press the power button and have three choices, the default one is "turn all the way the hell OFF". Power button, enter. Done. Or just close the lid, if sleep mode is appropriate. I wish the rest of the software industry could figure that type of design and elegance out. Have I run into Apple bugs? Yes. They're usually esoteric and weird, and rarely found in the "80%" part of the code, and once reported -- they usually DO get fixed. It usually takes a while but they're gone in the next major release. Are they still infuriating? Perhaps more-so. You get used to Apple's stuff "just working" and get more frustrated than usual when you find a real bug in a feature you wanted to use. :-) Nate -----Original Message----- From: piclist-bounces@mit.edu [mailto:piclist-bounces@mit.edu] On Behalf Of Rolf Sent: Thursday, January 29, 2009 7:20 AM To: Microcontroller discussion list - Public. Subject: Re: [TECH] Agile programming (was Re: [P|C] Banksel) Nate Duehr wrote lots of things.... : > On Jan 28, 2009, at 10:27 PM, Rolf wrote: > > Just as an over-all response to your mail: 1. the software we write does not do anything transactional (i.e. our software can not change dollars and cents, there is no risk that enyone could program our software to embezzle. 2. financial code is full of bugs. You should not have warm and fuzzies for even the best financial institutions. Always check your statements, etc. Still, the risk in finance of software errors causing you losses is insignificant compared to direct human manipulation. 99.9% (a number I have just invented) of money losses from clients/investors in financial institutions will be related to errors other than software. Further, software errors are typically discovered and fixed and the financial consequence corrected. Human errors are seldom dealt with in such a manner because your ciontract with the financial institution will have a disclaimer about that.... 3. Even though I say we write risk management software it does not mean that our programs manage risk... our programs provide risk-management data to risk-management people. These people understand the models we use to calculate risk metrics, and they understand the flaws in the models we use. The whole idea of financial modeling is that you simplify the real-world in ways that make the problem more easy to understand and quantify. The user of these models has to be aware of where the model is deficient, etc. This is something you should be aware of in any engineering task. Still, for the record, our clients have fared relatively well in the past. Then again, people who are willing to spend millions on licensing fees for our software are typically risk averse anyway and likely would make similar decisions without our software. Given that more than half of the world's largest banks use our software and we have not lost a client yet, I guess our clients are survivors. 4. as for 'the best', well, it is a subjective thing. The company I work for has earned the respect of the entire industry, and has a reputation of excellence. I am a part of that. Take it for what it's worth. 5. as for management whims.... well, they are. When a client wants to model a new type of financial instrument, and offers lots of money to make it happen, the whims of managers can change. Suddenly that performance problem you were engrossed in is not so important. 6. As for award winning software, the software we all write is award-winning too. In fact, in most financial risk categories our suite is acknowledged industry wide to be the best. 7. I guess all nose-ring wielding people are financially irresponsible? What, African Americans too?, how about tattooed people? That comment is despicable! Your spouse needs that tee-shirt. 8. The whole concept of Agile development can be applied in bits and pieces to any development. it is hard to determine what exactly makes a development process agile or not. Not everything that looks/smells/feels agile is. That is why I said I had experience with a diluted form of it. In fact, my understanding of agile is as fuzzy as the concept itself. 9. I believe software development is like a marriage... the two 'partners' are the product and the developers. Between the two of those you have to come up with a system that works, and, since every developer is different, and every product is different, every 'marriage' will be different too. Different things work for different 'marriages'. Agile development is a process that can be used to model the 'marriage' on, but it is not a one-size-fits-all thing. Still, there are many similarities between on set of developers and another, and one product and another. As a consequence, most 'marriages' end up looking, for the most part, very similar. 10. Why not use agile for our stuff? Well, we have 25 years of investment in our product, and it is written parts in Cobol, C, C++, C#, Java, web applications (asp, jsp, html, perl/CGI, javascript, ajax, etc.), Windows GUI's, X GUI's, command-line, batch, interactive. The biggest challenge is sometimes deciding whether to re-use a component or to re-write. In most cases, re-using makes most sense. 11. The same type of processes and people that launched the shuttle also launched the Mars Climate orbiter: http://en.wikipedia.org/wiki/Mars_Climate_Orbiter Overall I found your response to be more emotional than I would have expected. Is everything OK? Rolf >> I am solely responsible for certain functionality, and for other >> aspects >> > > That is surprising in financial software. You could add the famous > "shave off all the rounded numbers into my Swiss bank account" > function in your code, and no one would audit it? > > We can't even do that in "answer the phone line" code in telco -- > hearing that there's ever only ONE engineer in charge of a function in > financial code, doesn't give me warm fuzzies. > > (Yeah, I know they'd catch the above example, but you get the idea.) > > >> I work in a team as we together develop and maintain a single complex >> middleware application. On a company level we deliver a complete suite >> of financial risk management software components that interract/live >> > > I guess it doesn't manage risk all that well -- or your customers all > didn't lose 30% last year like everyone else? Hahahaha... > > Well, you did say you were "feeding code to the financial crisis", and > everything I've heard is that the mortgage stuff was quite heavily > affected by "computer models". Not that I don't hold the dolts who > BELIEVED the models responsible, mind you... but it's interesting stuff. > > >> Further, the company I work for likes to believe they have the best >> programmers (and I like to agree ;-). We are given a lot of freedom to >> > > That right there says something. The best rarely think they are. :-) > > >> do things 'our way', but the payback is that we have to be flexible, >> multi-skilled, and be willing to re-prioritize things on a management >> whim. I am one of many people in the company who have specialist/niche >> technical and business knowledge, and a broader overlapping system >> knowledge. >> > > If they're reprioritizing on a "management whim", the managers aren't > very good and don't have very good goal-setting skills in the first > place. So I wouldn't put too much stock in that "best programmers" > thing, since they're not very good managers to begin with. > > >> And worse, the product they developed has won all sorts of awards, and >> is well regarded in the financial industry, unfortunately it is a >> bugger >> to integrate with the rest of the system. It will probably be >> stand-alone for a while. >> >> In the end I guess you could say the company has a split personality. >> > > One that writes award-winning software, and one that doesn't? Heh heh. > > >> Development is scheduled from a business-functionality perspective >> (e.g. >> we need to build a Credit Risk Assessment strategy to calculate Risk >> Metrics on 'Toxic Mortgage Backed Assets'). The Finance Guru's will >> build models of what calculations need to happen (about 30% of our >> company has a phd, and many have 2 - not bad for 'programmers'), and >> the >> process will be broken down in to what data needs to be available, >> etc. >> From that point on a project manager will be assigned the >> responsibility of ensuring the software can produce the correct >> metrics, >> and will then start the battle of getting dozens of teams to schedule >> time to actually ensure that the databases, middleware, reporting >> frontends, calculation engines, etc. can actually process the data >> coherently. >> > > And meanwhile the bank using the software should have just turned the > idiot with the nose-ring and his girlfriend wearing the "I'm with > Stupid" T-shirt down for the loan... hah. > > >> Agile development would put together a smallish team to work on the >> whole thing from beginning to end with mini milestones, etc. This >> would >> not work for us because we often need to get many dozens of people >> co-ordinated to get the functionality available in many components >> that >> need to be backward compatible for hundreds of other Risk-Management >> processes. >> >> Basically the scale of development, testing, integration, and >> regression >> management preclude any one (small) team of people from having the >> time >> to learn what they need to know to get the job done. >> > > Isn't your job as you described it -- as an individual -- already a > "small team" within that bigger whole that gets those big projects > done? You do things you say the other's can't, you have your own > deadlines and goals... things change, you adapt... sounds "Agile" to me! > > >> While I am sure that I am replaceable (the whole hit by a bus >> thing), I >> also know that in the scheme of things, I have taken years to get to >> the >> point where I know the financial models, the computing models, the >> data >> models, and the development models well enough in our company to be >> able >> to produce top-notch and cost effective software. While there are >> people >> in the company that could replace me (and I them), a 'fresh' Agile >> team >> would take years to gain the collective experience of a bunch of >> people >> to get the task done, and not also run in to all sorts of >> compatibility, >> regulatory and other issues. >> >> An agile team will fail if it is required to disintegrate routinely to >> work on unrelated maintenance tasks. That's basically why it would not >> (does not) fly really well in the areas i work. >> > > This is why (and I don't like this model, but it's still commonly > done) many companies break up "Product Engineering", and "Continuation > Engineering" into different departments that don't co-mingle. > > >> Still, there are times when we need to build (small) new pograms >> that do >> whizz-bang things, and we get to do the especially fun stuff of rapid >> prototype development, and then it is close to 'Agile'. >> >> For the record, we got a new 'chief' recently up top at work, and his >> philosophy is that maintenance is not much fun, we should all be doing >> agile type development, and we should outsource the maintenance to >> Estonia. Personally I think it would be a mistake because maintenance >> should be the responsibility of the developer if only to ensure that >> they develop maintainable code ... ;-). >> > > How about pushing for code that needs LESS maintenance from the > start? Write once, use. :-) > > (Everyone says it's impossible, but we've all seen small projects > where it somehow worked out and the "thing" hasn't been touched for > years, and still works. I'm not a programmer by trade, but it seems > to me like Agile is all about just scaling that "small win" type > situation up into the building blocks for a bigger project, isn't it?) > > It *is* possible to write almost perfect code, after all... > > http://www.fastcompany.com/node/28121/print > > :-) > > Nate > -- http://www.piclist.com PIC/SX FAQ & list archive View/change your membership options at http://mailman.mit.edu/mailman/listinfo/piclist -- http://www.piclist.com PIC/SX FAQ & list archive View/change your membership options at http://mailman.mit.edu/mailman/listinfo/piclist