This is an attempt to provide a coded example that FunctionalProgramming (FP) proponents can use to show FP simplifying and/or shortening code outside of the systems software domain and beyond mere foobar examples. Although there are lots of different potential metrics to use, this particulate topic focuses primarily on CodeSize.

http://www.geocities.com/tablizer/chal06.htm  (NoteAboutGeocities)


That link includes:

* General description
* Runnable source code (link)
* Actual screen shots of app
* Actual screen shots of tables
* Actual screen shots of data dictionary (link)
* More details given below near PageAnchor more_details (below)

----
'''Summary of arguments''' (as I view them, you are welcome to present an alternative summary)

FP: FP significantly reduces code size

Top: I would like to see an example

FP: See, it can simplify your Example 6 loops.

Top: I could re-write them to simplify the loops without FP.

FP: But that does not properly close the database handles/pointers on error.

Top {Issue 1}: However, the challenge was to "significantly reduce code-size", based on an FP'er claim. You are moving the goal-posts. "Significantly less code" was never demonstrated for the given example (#6).

Top: {Issue 2}: I could reduce or eliminate the need to leave handles open by practicing SeparateIoFromCalculation such that the code-size difference is negligible.

FP: But SeparateIoFromCalculation has limitations...

Top: (Discussion then spins off to various topics about pros and cons of result sets versus cursors. I agree that result sets have limits in some cases, but they are rare enough I claim to not affect total code size much. The vast majority of loops are not affected by the drawbacks of result sets.)

''The HaskellLanguage forces separation of I/O and calculation because it is not good style to put calculation in monads except when you must.''

''It's not really about SeparateIoFromCalculation, it's about pluggable architecture. It's about "How can you create a function that, logic used deep down in the function calls, can be customized at the site of function call". You can not have this as easily without FP''

I would like to explore a specific need. That was the point of using challenge #6. If you have another fairly complete or complete example that illustrates such, I would be happy to take a look. But, I am skeptical of claims of heavy need for such without seeing for myself. Some of your coding styles seem so different from mine that the frequency may not be an issue if my style is assumed. (see PageAnchor: coding_thumb_rules, for example.) -- top

....

FP: FP better isolates variables to specific blocks

Top: I tend to agree, but 1) Problems caused by such "loose" variables are fairly rare and 2) it may be a language-specific thing that can be "fixed" without FP.

....

FP: Example is not a good demonstration because....

(I am fuzzy on exactly why it is being rejected.)

....

Top: My conclusion thus far: Demonstrated FP advantages shown so far are quite minor such that adding complexity to a language/design is not really justified, or breaks roughly even at best. I still see no slam-dunk evidence that justifies the FP bragging I have encountered. If you personally like it, I can live with that, but suggestions that it revolutionizes ALL software are off the mark so far.

-- top



''Why does''
* ''Making a function first class object that can capture the binding of its variable''
''adds complexity to a language more than''
* ''Add "if/while/for" construct to a language, where it's the only construct that can takes a block but your own function can't.''
* ''Add "and/or" operator to a language where it is the only kind of operator that have shortcut evaluation, which your own function can't''
* ''Add a syntax for List comprehension''
* ''Add syntax for Try/Catch''
* ''Having to break language syntax compatibility if the language later decide to add new construct such as Try/Catch, List comprehension where it never have before. (Note that in case of FP, it will also break language syntax if we were to have special syntax for some construct, however, with FP, most of the time, the new facility can be implemented as function, and it will still looks nice like a new syntax)''
* ''Likely break language compatibility every time we want to apply some programming paradigm (AOP, OOP, TOP, whatever) because implementing such feature as library does not integrates well with the syntax of the system.''
''?''

Well, this is sort of off-topic, but it seems you want an infinitely tweakable language. I might like that if I was the only reader and writer, but other people would just reinvent their language to fit their own head and consistency would go out the window. Every shop would have a self-rolled language more or less. If one could demonstrate significant improvements using such an approach, I might be more open to it.

By the way, I am not for try/catch blocks. SeparateIoFromCalculation reduces the need for that also. Some of those others I might cross out also. Basic list handling can be implemented via string libraries alone, no new syntax, and bigger stuff would often be best in a database.

  list="1,2,3,4,5,6";
  while(x = getNextInList(list)) {
	print(x);
  }
This can be implemented by simply chopping off one item per iteration in the getNextInList function (pop).

''I cannot believe you actually show that code. It's, IMHO, ugly... The list cannot contains object of other type (string, file, socket). One will never use that as a replacement for list. Did you think even a bit before showing this example? Or did you just want to put up something?''

* I generally don't have a need to put sockets into structures in my domain. I rarely open more than one or two at a time. (As stated near the bottom, you FP'ers have a bad habit of opening too many things at the same time. Maybe it is a common need for your particular domain, but not mine.) And, one can usually use an array or map for that instead if they wanted to do that. Maps are a super-set of lists more or less, thus we don't need a proliferation of built-in data structures. Furthermore, if a socket handle is just an ID, such as a socket/handle number, then it still can be put into such a list, especially in a dynamic language. It is often better normalization to reference stuff than to copy it into structures/objects and string-able ID's make such easier. Debugging with RAM addresses is a pain, if even possible for a given language. A relational brain seems to view this kind of stuff differently than you. -- top

''As for infinitely tweakable language, What's the different between allowing one to reinvent the language which fit their own head, or the problem domains, and disallowing it? What's the different between this two library?''

 re = createRegexp();
 g1 = createMatchManyNotRange('0','9');
 re.addGroup(g1);
 g2 = createMatchManyOfRange('0','9');
 re.addGroup(g2);
 re.match("hello 1234");
 if(re.matchGroup(g1)){
	// do 1...
 }
 if(re.matchGroup(g2)){
	// do 2...
 }

''and''

 regexp-do [
  /([^0-9]*)/ { do 1... },
  /([0-9])*/  { do 2... } ]
''I think the point that allowing one to build a language for their library is moot. Because, after all, when you are using a library, you are really using their new language, the API IS the invented language. The only different is that, in language that does not allow FP. the invented language looks ugly, you see kludge that shows how the new language doesn't get treated as first class language construct. It's like if you have to draw a picture, one system only let you write description of picture, while another just let you draw the picture. What's the different?''

I am not sure what your example is trying to illustrate. There are many ways to clean up reg-ex syntax/API's without using FP. We are wondering away from challenge #6 here anyhow. If you wish to start a topic such as FpMakesRegexBetter, be my guest.

''I'm not saying there is no way to clean up Regexp API without using FP. I'm saying that there's NO WAY to clean up Regexp API without reinvent custom language (the regexp part). Just like there is no way to Clean up Database query without having invented SQL language. It shows that there is no benefit of disallowing one to reinvent languages for specific domains. Because, after all, API call is also an invented language; the different is just that it's invented on the language that discourage inventing another sublanguage; That's why it's uglier than another.''

''Move Some to ApiIsLanguage''

Languages such as Tcl are pretty good at allowing/making sublanges without explicit FP, I would note.

''Yes, by using eval. However, we already talk about how using HOF is more suitable than Eval in DynamicStringsVsFunctional.''

I have not seem decent real-world/realistic examples there.

----
''Moved from TopMind. Maybe move it back when it is cooked, but can we barrow a topic for a few months to rant around with? Thanks.''

10/2004: Under HowCanSomethingBeSuperGreatWithoutProducingExternalEvidence and DynamicStringsVsFunctional, they kept claiming higher-order-functions or FP can produce '''significantly less code''', but could not prove it using Challenge #6 (see link below).

''Lies.''

{Damn! My evilness has been exposed. Oh the shame!}

Nor, did they clearly state why their techniques could not shorten #6.

''More lies.''

They made some vague insult about it having SQL in it, but refused to clarify. All talk and no walk. I am holding you to your claims, guys. No wiggling allowed on my watch. You simply '''failed''' to deliver less code for #6, and without a good explanation.

''You were shown many times, you just didn't understand it.''

'''I don't have to "understand it" to know whether it is significantly less code or not.''' A quick eyeball scan will tell that. I made a big deal about the claim because such is something that has an easy-to-see metric rather than just another LaynesLaw battle. This time you can't pull the old "you would see it was better if you were smart like me" escapist trick.

It may later turn into a battle over how to score code size, but you have not presented anything to compare yet. Since you claimed "significant", there should not be much incremental bickering about the code size metrics. I suppose you could also offer fake code to trick me, but the challenge is to produce signif. smaller runnable code, not to fake me out.

Can I rightly claim that you walked away from the challenge?

''No, but you can rightly claim ignorance.''

[ ''Indeed, he can. TopMind was shown the use of higher order functions to make new control structures many, many times. He was shown how it could shorten loop-oriented code, how to extract regions of code out and make them into HigherOrderFunction''''''s that handled common tasks, and how to even use common HigherOrderFunction tools to make domain sublanguages.''

''Top's ultimate response was that, in the face of for loops, case statements and the IteratorPattern, all of these things were rendered pointless. Even when his approach was dutifully extended to show it was several times larger than the counter-examples using HigherOrderFunction''''''s, he still claimed they were not practical or didn't offer a net code savings.''

I don't understand this. Please clarify. If you mean the "weather example", I have insufficient info about the requirements to compare. Challenge #6 is better documented and I supplied runnable code. You did not. '''Runnability is a fact, not opinion'''. Further, I offered to answer ANY question about #6, but the reciprocal for the weather example was not offered. If you want to use the weather example, then please do it right and finish it.

''Quite interesting to notice how all the code samples that proved you wrong, have somehow disappeared from that page.''

* I did not delete any examples, I swear! Did you check in ArrayDeletionExample, by the way? -- top

''Apologies, I had the pages confused. Yes, ArrayDeletionExample pretty clearly proves the less code claims.''

* Good! Apply them to #6 to make "significantly less code" and you are done and win. (Where are these alleged "pretty clear proves" anyhow? I did not see any that did not collapse under scrutiny. I saw '''nothing''' that was more than about a 5% code-size difference. That does NOT qualify as "significantly less". Somebody is delusional.)

''[It's amazing how carrying an axe with the intent of grinding it can distort your vision.]''

Yes. It made me completely not see your Great Short Version of #6. It is totally invisible to me, as if it never existed. FP fanatics are not exactly mainstream either. So what's with this "axe" talk? Glass houses.

----
Top, you like to point out challenge #6, but whenever we point out challenges, you seem to take delight in claiming that is "not your domain." What I don't understand is why this works for you (in that you can claim TOP is superior, say it is only for certain domains, then act like it's better for all of them) but not for anyone else?

How about this. You're the one with something to prove, TopMind.

''No! YOU claimed your techniques result in significantly less code.''

Table Oriented Programming is a simple technique, really a footnote in the collective consciousness of programmers. You use it when it makes sense to use. You're claiming it always makes sense.

''Did not. I don't claim my favorite techniques are optimized for all domains. Do you?''

So why not tackle http://www.frank-buss.de/challenge/index.html which SlashDot recently posted. See if you can shorten it. You're competing with PascalBourguignon's version.

You up to it?

''Why not have him compete against Dirk Thierbach's 6-line HaskellLanguage version, or Konstantin Metlov's 3-''character'' JayLanguage version?''

* Hmmmm. Isn't J generally a declarative CollectionOrientedProgramming language? Sounds almost TOPish. Collection-orientation won, not FP. (Maybe use some sort of CartesianJoin on or for all unique combinations of 3 points? Why no Prolog solution there?)

** But before screaming "VICTORY!" note that the actual solution is mathematical and the program just calculates (3^n). It uses no collections. The Haskell version on the other hand does count triangles, and also works for irregular graphs, where the J version has to fail.

** ''See ExampleSizeIssues''

Sure. Compete against any of the good ones. As long as the feature set is comparable, it's fine.

''Challenge #6 existed first. Stop changing the challenges until you find one that breaks TOP or is outside my scope of familiarity. Can't FP improve anything beyond university math-lab toys? '''If FP cannot reduce 6's code size, then I consider your claim null and void*.''' Now code or erode!''

[*] Unless you come up with a detailed justification for why your FP tricks don't work on that particular example. If FP has limits, lets find and document them. There is no reason to be afraid of such, is there? Besides, a language tuned for lab toys may be completely different than one tuned for real business projects.

----
<rant>
Stop arguing about FP, TOP, I now present you the new language and challenge, my language is called writing CRUD-SCREEN-REPORT-001 language. It's CRUD screen for Employee and department oriented language. It has only one line of code you can do with it. The code is

 run().
This code start up the database server running 24*7, with full logging, and administration for The company name taken from Machine environment variable.

Other flexibility is out of my language's scope.

Now this is my challenge, you TOP mind please do coding that is shorter than this in your language? Or else your claim of TOP is null and void*. And don't challenge my language and paradigm by other algorithm, 'cause that is out of my language's scope.!!!
</rant>

''Is there a point to this? Hints of DomainPissingMatch.''

Sure there is, I'm tired of seeing top, saying "No, that part of the problem is not in my application domain." He tried to say HOF has no benefit over his TOP oriented, but when someone showed how HOF is used to achieve simplicity in some problem top can not do, he would just said that it was not in his domain. He is not trying to learn anything. There may be smug lisp weeny, But even those guy knows about what ever programming practice you are talking about, Top is just not so. Me as a CRUD-SCREEN-REPORT-001 programmer, I would just said Top things are useless because, I do all I want in just one line of code and I don't wanna learn anything new.

* Perhaps the above is suggesting that libraries are doing all the work, and thus it is not a fair comparison. But doesn't Lisp have libraries with HTTP wrappers, etc? The table browsers are not really key to the demonstration, so don't get side-tracked trying to duplicate table browser capability. If it is later an issue, we can separate it.

May be TopMind is not Top at all.

''The "significantly less code" (SLC) claim did not exclude CrudScreen apps. If you want to revise your claim, be my guest. Something about #6 seems to get under your skin such that you change the subject. It is hard for me not to conclude that you are afraid of something in #6. The original SLC claims were written with such certitude. That certitude is quickly draining away, like a fast-talking street racer who is finally pressured to race his car instead of just talk about it.''

Why does anybody keep giving this crank airtime? Seriously, just go read his pages. Rant, rant, rant. OO is evil. OO proponents are misguided. FP people are ivory tower theorists. No-one recognizes my genius, because there's a ''conspiracy'' by OO tool developers to market their stuff. No, really, read this crap. He goes on and on about requirements, but can't write coherent requirements documentation for this or any of at least five other challenges. Most of the pages don't even describe what the challenge is.

''Content outside of the "challenge" page is a separate issue. Somebody claimed higher-order-functions or FP can produce significantly less code on this wiki and I will hold them up to that claim. My rants and their claims are orthogonal issues. One side being an alleged rant-head does not dismiss the other side from providing empirical evidence of their claims. Note that others may be interested to see the FP claims backed with code despite what the "ranthead" says about it. The results live outside the ranthead.''

Re: "Most of the pages don't even describe what the challenge is"

Unfortunately, those who initiated the challenge are the ones responsible for providing one. What you are reading is the response, not the initiator. But, in the case of #6, there is a reference implementation to use as the "requirements".

''Questions can be asked and answered below.''

----
I don't understand this library size issue that keeps popping up. Will functional need less libraries? Is it easier to make libraries with functional? Given the same or similar libraries, will the functional program be smaller? If there are too many variables (factors) to compare, then how does one know if a functional version will be significantly smaller? The fault in the alternative may be the design or quantity of the libraries, not the paradigm. It almost sounds like the functional side is implying that if one has good enough libraries, one does not need functional to reduce the code size or code complexity. Good libraries seem to be the GoldenHammer factor affecting code size according to this argument and not functional. It is the allegedly great or thorough or domain-specific libraries that made the code example small enough that functional cannot reduce the size by much. It seems the reader is being sold on libraries, not functional.

----
The problem with what TopMind is asking is that it largely misses the point. He's basically asking us to take one tool from our toolbox and use it to attack a small application without considering other tools. Using AnonymousFunction''''''s and the FunctorPattern in your code is a very significant win in some cases (when you work with collections), but were I to attack Challenge #6, I would use many more patterns. Certainly, AnonymousFunction''''''s iterating across collections would be one of them, but...

''You are allowed to use as many FP features as you want.''

I think TopMind is trying to compare Apples and Oranges here, and that's why our examples have fallen on deaf ears. TopMind considers TableOrientedProgramming in its broadest sense, whereas the one point of FP we argued on (acting upon collections) is but one of the tools of the larger FunctionalProgramming picture.

While it should be easy to beat the line count of the given example (you don't need anything fancy to beat Challenge #6 there, the design is... very conservative) I believe it wouldn't prove anything to TopMind. Until he learns FunctionalProgramming and groks it, the advantages of such a system will be lost on him.

''As explained in HowCanSomethingBeSuperGreatWithoutProducingExternalEvidence, I reject the idea that something can be significantly better without being able to give specifics. The "if you just knew it as well as I did, you would feel the benefits" is absurd. The fault is your articulation skills in that case, NOT my understanding. Everything objective is demonstratable in smaller examples. I agree that smaller examples sometimes are misleading when applied to larger ones, but at least they can demonstrate patterns to be found in larger stuff. We may later debate about the frequency of those patterns compared to real stuff, but at least the patterns (lower case) themselves are demonstrated so that we can then move to the frequency issues.''

There are tons of external evidence to support the merits of FunctionalProgramming paradigm, you just pretend not to see them or refuse to look for them. Don't confuse your ignoramus status with the lack of evidence. See IcfpProgrammingContext for example, let's see if using your TableOrientedProgramming you can even attempt to solve any of the examples out there, much less to compete with FP hackers out there. If you complain that those challenges are too elaborate for you to try, I have simpler challenges for you like EnumeratingRegularLanguages, or any algorithmic problem. Take the recent one proposed by somebody here, like SicpIterationExercise. Even as the last one is very amenable to TableOrientedProgramming, a Haskell or ML solution will still be much more elegant, but let us see your FoxPro solution first.

''I presented a challenge first. But, I will take a look (when it is ready).''

What was your challenge? If I'm not mistaken it was no coding challenge, it was a request for examples where FP is the best approach, so I pointed you a bunch of them, and I can point to you tons more, so please tone down your handwaving in the meantime.

''I moved the link to the top.''

Given the link you posted, this page should be deleted as it has no value? What's the purpose to compare challenge six against FP? None whatsoever as far as I can tell. FP is not designed to substitute SQL, nor is it intended to solve all problems int he world.

''Can it help the parts that are non-SQL? If not, I would like to know why. That might tell us something about the nature of FP and where it helps and where it does not help. We may all learn something from such an answer. The original "significantly less code" claims did not have caveats for specific domains.''

If an environment (be it FoxPro or DelPhi or VisualBasic or whatever) already offers you precanned components to interact with a SQL database, run reports, etc, then that's the deal. FP was not designed to solve all challenges in the world. It's like discussing that differential geometry is not adequate to solve diophantine equations (in case you wonder the two have nothing in common).

''The "precanned" stuff is mostly just library API's. You can assume or make the same API's for your example if you want. If API's are the secret to competing with FP, then why sell FP? The "magic" is in powerful API's, not FP under that logic. I am thus confused about your API complaints. Regarding your diophantine analogy, if FP's benefits are limited to certain domains or techniques, '''let's get it out in the open.''' I would probably solve most problems I encounter for any domain with TOP techniques. Performance, lack of tools, or management paradigm mandates are the only thing that would stop me. If TOP is a flop for other domains outside of performance or microsecond timing issues, I would like to know about that also. (AreRdbmsSlow)''

So bottom line is: you try to mix unrelated things in the same bag. You challenge six cannot be used for any discussion whatsoever about FP, and all your comments related to FP on the basis of this challenge are to be discarded. If you are honestly curious about what FP is good for, try to learn first, and handwave less.

''I want to know why they are allegedly unrelated.''

Because nobody targeted FP either in general or in a particular implementation language to address your particular domain of interest (boring business applications).

''See DomainPissingMatch and AreBusinessAppsBoring.''

Instead FP shines at solving mathematical and algorithmic problems, while producing high quality (low bugs, low maintenance code). FP environments in general don't have precanned libraries and WhatYouSeeIsWhatYouGet IDEs to make them compete with VisualBasic, BorlandDelphi and others in slapping together GUI screens and database queries.

''Um, that is a web app. HTML, basic HTTP/CGI-like libraries, and a little JavaScript. There are some GUI screen shots, but they are mostly just to show the ControlTable content. '''I purposely stuck with a web interface there to avoid a GUI library war'''. (BeenThereDoneThat and I agree with you that it makes comparing difficult.) BTW, what is an example of an "algorithmic" problem?''

[Actually, there're several decent libraries available for FP languages, eg. WASH for HTML/CGI, Fudgets for GUI, and HaskellDb for database access. Unfortunately, most of these are student/hobbyist projects, which means they pay lip service to conveniences like easy installers. I spent 2-3 days trying to get Haskell and WASH installed, and I haven't even tried HaskellDb because of the long list of dependencies. Preliminary investigations are encouraging though; the equivalent of RptList.asp was half as long (10 lines vs. 19). I expect bigger savings with the main program, as I can utilize HaskellDb's composability of queries and Haskell's PatternMatching features. -- JonathanTang]

* Regarding the "10 lines vs. 19", I did not use the techniques under QueryAndLoop, which would have shrank it.

Although it could be done in principle (and some disparate efforts proved that high quality components), no big company invested the necessary money in making FP a success story for this domain.

''So one should not use FP for biz apps???''

So your challenge 6 is completely futile and borderline trolling: nobody pretended FP is the optimal solution for the domain, there's no reason why FP ought to be the optimal solution for that, nor the fact that FP currently isn't the optimal solution for that domain detracts in any way from the value of FP.

''Again, the original claimers seemed pretty adamant and intense.''

Therefore the very title ChallengeSixVersusFpDiscussion is borderline idiotic. To drive the message home I can make ChallengeSeven: write a good sonnet about wiki. And then create a page ChallengeSevenVersusChicagoManualOfStyle, and then '''let's get it out in the open''' that ChicagoManualOfStyle does not help you write a good sonnet. And then let's criticize the well-respected writing guideline for the reason that it doesn't cover writing sonnets. 

That would be the same thing as you do here.

''I am still not getting your analogies. Biz apps are not outside mainstream. In fact, I expect offshoring to hit physics-oriented stuff harder than biz because Newton's laws are the same in Bangalore, but business culture is not.''

Not quite. The thing is, physics is ''already'' heavily offshored, because Americans never had a monopoly on smart physicists to begin with. The bulk of physics discoveries have come from other countries, and then their discoverers have moved to America to make more money afterwards. As a result, physics is already a pretty egalitarian playing field, where you need to be smart to do it in the first place, but it doesn't matter whether you're a smart American or smart Indian or smart Chinese or smart European.

We can probably expect biz apps to tend towards the physics situation as what you know becomes more important than where you're from. Sucks for the LessAbleProgrammer''''''s in the U.S, but it's great for all the computer whizzes in India. And you could argue that this is how it should be. -- JonathanTang

''Perhaps the offshoring stuff should be moved to a different topic.''

----
You can prove this by looking in ArrayDeletionExample. We took his code examples, extended them out, and then showed how a FunctionalProgramming approach could reduce code count and increase code clarity. It even allowed us to give arbitrary verbs to collections (he immediately noticed we use database verbs frequently). But, despite the simple comparisons present on that page, TopMind doesn't get it yet. This is most likely because he can't envision and entire program designed around such principles (which is hard to do unless you've tried it), so let's not get too angry at him.

''I don't know why you guys keep defending ArrayDeletionExample. I have read it multiple times (and clarified some old points in it), and don't see any magic. I reworked some of the code to be more competitive size-wise, I would note. '''Nothing is more than a few percent smaller in the end. You have not shown any slam-dunk size reduction.''' The weather example was admitted to be limited by a Non-disclosure Agreement. Thus, key questions about why it has such odd design constraints cannot be answered; and this makes it useless for detailed comparisons and analysis.''

Top, you need to understand what we're talking about. FunctionalProgramming is a separate but related paradigm to TableOrientedProgramming. It's very hard to see the forest when all you're doing is focusing on one tree.

''Sometimes I think you FP'ers are fixing the wrong forest. Why do you keep implying the problem is me instead of just show obvious improvements? I don't get some of you guys. You're so evasive yet make such strong claims. Are your claimed benefits objective or subjective? Make up your mind.''

You make up your mind, Top. You're currently full of BS clutching at straws (i.e. irrelevant petty examples), when you have been directed what it is you need to look at. Let me repeat: IcfpProgrammingContest, and algorithmic problems like EnumeratingRegularLanguages, or if you whine that those are too complex you can try even something as simple as SicpIterationExercise. Put your code where your mouth is or just shut up for a while, and stop polluting wiki with repetitive ramblings. You cannot whine continuously that nothing has been shown to you, as long as it has been shown and you haven't done your homework.

''My challenge was first. I turned on the TV, so why do you get to keep changing channels? Besides, those are university lab toys. I want to see something that helps for biz apps, not university lab projects.''

Your challenge was first to be irrelevant as it has been explained to you.

''Your explanation was unsatisfactory. They don't depend heavily on libraries. You are flat wrong about that.''

The other are real challenges, as a proof of that anybody who knows an if then else can write your "biz apps",

''DomainPissingMatch talk and ArgumentFromIntimidation. If they were so simple than FP should knock them out of the ball park during the first swing.''

IcfpProgrammingContest challenges are for GrandMasterProgrammer'''''s only. Your exclamation "university lab projects" sounds very much like '''the grapes are sour'''. Why don't you show us that you can solve at least the simplest one so that we can appreciate your mastery of the ArtOfComputerProgramming.

''The sour grapes are on your side. My challenge was first, Bucko! Further, most of those appear to be contests where the programs compete with each other rather than fitting requirements. In other words, more like making a program to play chess rather than make a chess game. That makes it too difficult to compare. We want to study and compare code here, not output.''

Whether biz apps are boring or "easy" is irrelevant. Can FP significantly reduce the coding size (or whatever objective metric you propose) for them or not? Just answer the frippen question with biz code. Stories about where "real programmers" hang out is off topic and smells of social intimidation. I don't challenge FP for making chess playing software, that is just not the issue.

----
On the size reductions of Top's new code in ArrayDeletionExample, specifically Mandy and MandyTwo, TopMind totally missed the point of what the code was doing. I think part of the reason he isn't seeing how cool HigherOrderFunctions are is because he's not thinking in both runtime and compile-time terms.

If you look at his MandyTwo code, what he did was parrot a HigherOrderFunction example with a pair of named functions, offering none of the flexibility. This suggests to me Top doesn't get how you can dynamically alter behavior with HOFs, trivially. It also suggests to me he thinks of applications in a very web-application-oriented sense. You write something, and it's done. People don't really extend it, and if they do they don't touch old code so much as make entirely new regions. This mimics the development of websites. 

* Your characterization of web-site stuff is naive. Everything changes in the biz world. Further, I have been developing since the mini-computer days. I am not a newbie and have seen changes to software many times.

If he could see how a program uses HOFs to control behavior by managing code like data, placing HOFs at will all over the place, then he might be more interested.

This can be used even in software that traditionally centers around tables and doesn't care much for speed. For example, check out this procedure that you might find in a weblog (in a pythonish, rubyish, phpish pseudolanguage):

  define output_entries( blog_entries, output_type=Outputs::HTML )
	for each entry in blog_entries do
	if( output_type == Outputs::RSS )
	output << RSSEntry( entry )
	elsif( Output.type == Outputs::HTML )
	output << HTMLEntry( entry )
	else
	output << TextEntry( entry )
And if someone wanted to add new types, they'd have to extend this further. This is exactly the approach that TopMind said he espoused, and you can see on ArrayDeletionExample. This is basically a glorified case statement. It's annoying, hard to extend, and prone to breakage. Let's rewrite it with a HigherOrderFunction in mind.

  define output_entries( blog_entries, output_func )
	for each entry in blog_entries do
	output << output_func( entry )
Now, if we were to add a few HOFs, like the following, we won't see a massive lines-of-code savings right away:

  output_as_html = { |entry| return HTMLEntry( entry ) } # HOF that returns text of HTMLEntry on its argument
  output_as_rss  = { |entry| return RSSEntry( entry ) }  # HOF that returns text of RSSEntry on its argument
  output_as_text = { |entry| return TextEntry( entry ) }  # HOF that returns text of TextEntry on its argument
Or we might see it used inline, as is popular in Ruby and Lisp:
  output_entries( blog_entries, { |entry| return HTMLEntry( entry ) } )
We can invoke output entries with whatever we want. We've saved a little code, but not too much. Within 5%, as TopMind says. But, what happens when something changes? For example, it turns out our users are reporting that every so often, bogus entries are coming up in their blogs. Maybe our master database queries are broken somehow? But we can't easily reproduce this problem on our test server. So what we need to do is put a log on the live system and hope we catch the bug in action.

Well, with the original case approach, this would suck. We'd have to go through each entry in the case statement and add the same code over and over again. Cut'n'paste code at its worst, and clearly it's a pain to handle. But if we had the higher-order-function version, it's simple and painless and reliable!

 # Assume that the script parsed the URL above, and set "desired_output" as one of 
 # our output_as_* functors.

 output_entries( blog_entries, { |entry| debug << DebugInfo( entry ) ; return desired_output( entry ) }
See? That's ''much'' less code. Using HigherOrderFunctions makes your code more flexible. Imagine if only ''some'' of the blog sites were breaking. Since you might have thousands of users, you might want to be able to dynamically enable this logging to save on disk space and I/O bandwidth until they report the problem. Again, this would be easy with higher order functions, but much harder with the case-like statement. You'd have to add if-statements into every if statement, massively increasing the size of the code.

Flexible code leads to long term savings. Even though it wasn't immediately obvious, the design paid off when we had to modify our code. This kind of savings often comes when you least expect it. It's why when I wrote my Weather app (I'm the guy who talked about it), I used them. This is not to say they're a GoldenHammer. It's undeniable that they don't save much code in the short term, and that they are often more complex (especially in languages that don't support them well, like C++). But even if the code savings is minimal, the ease with which someone familiar with the pattern can extend the program shouldn't be taken lightly.

You might argue that "Eval" could mimic this behavior, and you'd be partially right. Certainly, eval does something very similar to what HigherOrderFunctions can do. But you're using Eval at the wrong time. Eval is a very slow and powerful tool. Really, it's meant only for use when you have no other option. HigherOrderFunctions have strengths ideal for this, and they can encapsulate eval calls at a later date, if you feel you need them. Claiming that you'd forego HOFs here for Eval is like saying you can't be bothered to take the parking break off your car, it's so much more convenient to just drive with it on, who cares if it ruins your gas mileage and break, right? Cars are getting better all the time.

I hope you can see were we've been going with this discussion now, Top.

'''Reply'''

First of all, why are you assuming that each output "type" is mutually exclusive? (Be very careful using the word "type" around me.) Is it not possible that multiple output "types" may be needed per request? You talk about making stuff be change-friendly, and to be change-friendly we cannot (overly) hard-wire the assumption of mutual-exclusiveness into the design.

A CollectionOrientedProgramming viewpoint would be to have the output formats (code snippets?) in one table/collection, and the input in another, and then join them as needed. Going from a one-to-many relationship to a many-to-many relationship does not require a change to the original output format table, only the introduction of an intermediate many-to-many table.

But, let's stick to a case-statement comparison right now.

Second, you claimed, "We'd have to go through each entry in the case statement and add the same code over and over again." I don't see why this would be the case. Let's assume a statement like this:

  function outputMessage(msgRecord, outputFormats) {
	result = "";
	initialization_stuff...;
	if (listContains(outputFormats,'html')) {
	result .= do_html_stuff();
	} else if (listContains(outputFormats,'rss')) {
	result .= do_rss_stuff();
	} else if (listContains(outputFormats,'text')) {
	result .= do_text_stuff();
	}
	post_processing_stuff...;
	return(result);
  }
We can put the tracing at the beginning (near "initialization_stuff") or at the end (near "post_processing_stuff"). I see no reason to put it under each of the output format blocks unless it is somehow specific to that format, in which case you have the same issue to contend with. Either it is sharable code or specific. Specific goes inside the decision blocks, general goes outside.

I also assumed orthogonality here (an easy change from a case list) by using a potentially multi-value list and also just appended the results to a string. I don't know what would be done in practice because I don't have the full requirements. If they need to be separated into different files or records, that does not change the general strategy here.

	general_pre	// common to all
	block A  // stuff specific to A
	pre_A
	process_A
	post_A
	block B  // stuff specific to B
	pre_B
	process_B
	post_B
	block C // stuff specific to C
	etc...
	end blocks
	general_post
If we follow SeparateIoFromCalculation, then we can intercept the results (or pre-sults) outside of the format-specific blocks.

As far as your weather example is concerned, like I already said, your claimed limitations are very suspicious. (See "perfect storm" comment under ArrayDeletionExample.)

Further, perhaps this discussion goes under ArrayDeletionExample instead of here. However, that topic is getting too long.

-- top

SwitchStatementsSmell contains somewhat-related discussions about case statements.

----
Top: If you factor out all the "template filling in" stuff, how many lines are left actually doing real work? It seems to me that this problem may be difficult to reduce because it's essence is trivial once all of the template code is ignored. HelloWorld usually has a "smallest correct value" in any given language that no amount of cleverness can reduce. If challenge six gets most of its bulk from the HTML and SQL code that is being used essentially as a template, the rest may not be reducible because it doesn't do much. In short, in your attempt to come up with a non-trivial example, maybe you just made a long trivial example.

''I do not know what you mean. You may find the example boring, but that is irrelevant. This is not an entertainment contest. What are you calling "template code", and how does that differ from non-template code? I always consider template code as something that you copy-and-paste and then fill it for specific circumstances. This is not that (although it could be used that way if one does not want to extend it directly, but so can anything).''

I mean code to fill in the blanks in SQL and HTML statements. To take a trivial example:
  print "Hello " + name + "."
Replace "Hello " and "." with three page long fairly non-redundant strings, and you will have a six page program that just can't be shortened (ignoring exotic things like data compression).

''Maybe that is because SQL and to a lesser extent HTML are doing most of the "work". I don't see that as a bad thing. It is a problem-solving approach: use high-level tools to do the work for you. If it is not something that FP can simplify, then we have learned something and the "signif less code" claimers should put caveats into their claims.''

''I am looking for techniques to simplify my domain, not university homework assignments that find the shortest path to all combinations of pi, etc. I chose that example because it is a bit closer to my domain than most of those offered. (But still not a perfect example. A better example would probably have more seemingly arbitrary business rules tossed in.) If you have a biz-oriented example you wish to present, that would be great also.''

* I notice that CollectionOrientedProgramming languages often beat FP languages in university-like contests.
** Which CollectionOrientedProgramming languages are you thinking about, and which contests? I don't know a single university-like contest that's been won with SQL, J, or APL. The normal high performers are Haskell and Ocaml (functional) and C++ and Java (OO). Strangely, Lisp doesn't seem to do all that well, which might be because macros don't make much of a difference until you have a large body of code. -- JonathanTang

Since neither HTML nor SQL are particular functional, that doesn't strike me as much of a concession. I'm not a big FP guy, but I'm happy to take a crack at it. How's this for a caveat: "Functional techniques are of limited use when you are forced to use a language that does not support them, such as HTML or SQL."

''Are you suggesting you can create signif shorter code for the same thing if you did not use SQL? Be my guest to try. (I don't think SQL is the ideal relationallanguage, but it is generally compact for what it does.)''

Not exactly. Although I think it may be possible, I suspect that challenge six is just too trivial to matter. Let me try to translate the idea into "structured programming" terms. Imagine that someone is trying to claim that subroutines can reduce code size. Imagine that someone else says "Prove that subroutines can shorten code by showing how it can make the following significantly shorter":
  for(i = 0; i < 100; i++)
	printf("Hello, world!\n");
  for(i = 0; i < 100; i++)
	printf("Wheeee!\n");
The program just doesn't do enough for subroutines to be of any use in shortening it. Subroutines, recursion, data structures, garbage collection... they just won't help. However, they are all capable of vastly reducing code size in more complicated programs. Does that make sense?

''No. I would like to see an example from a "complex" business application. The trick is to not let something get too complex by dividing it into individual tasks. The large-scale communication is via the database, not so much algorithms. More about this in ProceduralMethodologies. And, I do heavily use subroutines in #6. Anyhow, as far as simplifying the above:''

  repeatPrint("Hello world", 100);
  repeatPrint("Wheeee!", 99);
But let me get this strait. '''You are agreeing that FP won't help much with challenge #6 code-size-wise. Correct?''' Any other PF fans disagree? -- top
* Not exactly. I am keeping an open mind as to whether or not it can help. I am simply offering a possible explanation for why it may not be able to help, namely that the program just isn't complicated enough for the techniques to be of any value. My subroutine hypothetical was an attempt to draw a parallel. Since I can tell that you understand the value of subroutines, I was trying to explain how it is possible to create a problem so trivial that they won't help with it. It isn't exactly an argument, just an analogy to try and explain the idea. Also, if you include the definition of repeatPrint I doubt that you will get much if any code reduction.
* ''You agree that it is complicated enough to make use of functions/routines, but are you saying that it takes another "level" of complexity to make use of FP? I wish to see at least part of a biz app that uses FP for signif reductions.''
* Well Top, remember one thing. Unlike you, we're not fanatical GoldenHammer users. Even were we to say, "In this particular type of app that you chose, without any requirements for modification, FP provides minimal code count benefits." (and I am not saying that) then our position wouldn't really be diminished. Functional programming is just one of many powerful tools we keep in our toolboxes. So is CollectionOrientedProgramming, as well. Even you have pulled the same trick, implying that my weather application example was a PerfectStorm and thusly non-representative of normal coding activity. -- DaveFayram
* ''The person(s) who claimed "significant code size reduction" sure the hell sounded like a GoldenHammer proponent to me. If you are a mix-and-match type, then perhaps you are not the audience of this topic. I never claimed my techniques offer significant objective advantages (unless perhaps consistency can be measured), just not the reverse. Programming and software maintenance is mostly about psychology, not machines.''
* Programming is also about objective quality like EconomyOfExpression, EconomyOfExecution and other properties that are beyond psychological BlahBlahBlah. You sweep everything under the rug when try to say every technique is more or less the same. FunctionalProgramming does present distinct advantages for algorithmic problems and it has been shown to you how you can convince yourself of that.
** ''I'm not sweeping anything. I am asking to see actual reduction in runnable code using semi-realistic examples. Where are they? There's nothing to sweep yet. -t''

* ''You are right, performance is one objective issue. However, EconomyOfExpression sounds like another fuzzy holy war. I have not seen any distinct advantages beyond perhaps speed of your examples. I gave you speed already. This is not about machine speed, but the "less code" claims. If you produced one, I missed it. If you want to keep bragging about FP GoldenHammer, you better produce some convincing evidence, because I and many others are NOT going to take just your word for it. It seems you expect us to just to do that out of sheer force of evangelism. See also PsychologyVsCutAndDryPoll.''
** Top, can you have at least one conversation without distorting the opposite argument? Cut the BS, please. Nobody here claimed FP as a golden hammer, nor was it implied, but for certain domains it was claimed that FP is the best tool available, and to that you only contributed with a lot of handwaving.
** ''Perhaps there is confusion over what is being called "algorithmic problems". Do you  mean the algorithmic-heavy portions of any app, or a specific domain? I would like clarification on this term.''
** Perhaps claiming confusion is your favorite conversational subterfuge. If you try to solve at least part of the examples that have been provided you'll get a pretty good idea what "algorithmic problem" means.
** ''For now, it appears your FP code-reduction claims are not global, but limited to a "kind" of domain and/or problem. Without getting into a LaynesLaw battle over what that "kind" is (not in the mood right now), is it sufficient to say you are not claiming code reduction for all or most domain/problems? That's all I want to know for now. I don't dispute FP may have areas that it shines. I just want to know if your claim is global or not.''
** Perhaps you should learn to read better. If that's all you wanted to know, you should have known it a long time ago, if you bothered to read what people other than you were trying to say on this and related pages. Of course nobody made any "global claim" or whatever you wanted to name such a claim that FP is the best thing every time everywhere. If you still wanna play HumptyDumpty or LaynesLaw about "algorithmic problems", let me simplify it for you: '''whenever you write a non-trivial algorithm and it hurts''', remember that FunctionalProgramming concepts, techniques and practical languages might have helped you.
** ''I will believe it only when I see it in code, side by side with the alleged lame, repetitive, wilting spaghetti procedural/relational version. Also, "non-trivial" and "algorithmically complex" may not be the same thing. Many complex problems can use data-centric or declarative techniques. In fact, most of them are Turing Complete. Maybe you are just solving them using algorithmic techniques instead of declarative techniques. I am skeptical that university lab toys are a good test for real-world apps. Challenge Six is a at least as good a test for real-world stuff as the lab puzzle toys. It is almost like testing a business computer by how fast it calculates pi to 1,000,000 decimal places.''
** {I believe it is fairly common for RelationalWeenie''''''s and SQL-gurus to claim that a good many "complex algorithms" can be reduced to fairly simple queries if the schemas are "well-designed", views can be created, a powerful query language is available, etc.}

* As for business applications, well, yeah you wouldn't want to substitute VisualBasic with Haskell to slap together 3 update screens and 2 reports. '''However''', here's an example of practical application of functional ideas to real (and non-trivial ) business problems. (Moved to FinancialContractExample)
* ''Just because a product made around a concept runs does not mean it is necessarily superior software. Assembler and COBOL make programs run also. But, I will take a look at said paper.''

----
I disagree. And I'm working on proving you wrong. Unfortunately, you didn't even see fit to include an SQL schema with challenge 6. Urgh. -- JonathanTang

* ''As far as I know, the schema of every table is available on the web-page or links given. I am not sure if the type and length of every column is given, but that is mostly irrelevant. (I wish we had dynamic relational where column types didn't exist.) -- top''
* I think you are wasting your time for an unworthy cause, Jonathan. However, let us where it goes.

The schema is shown as screen-shots near the bottom of the page, and there is a link to the data dictionary page which shows example data. The "reports" table only shows the column headings, but the types are easily guessed. The "reportID" and "rptSequence" columns are integer, the rest are character/text. -- top

Yeah, my complaint is more that I have to take data_dictionary.txt and manually convert it from comma-delimited to SQL, which is a huge pain. I tried mdbtools on the Access DB, but mdbtools's makefile barfed on my machine for lack of cpp. Oh well. Just expect it to take a bit longer. Also, your link to Jim's Ruby version is broken, and I'd very much like to see it, as I can read Ruby easier than ASP/VBScript. -- JonathanTang

''I indeed wish there were an open-source MS-Access-like tool for quicky DB RAD stuff. Jim's server croaked and he never got around to recreating it IIRC. But he never implemented most of it anyhow, only created a rough framework.''

----
Apparent reasons given for not producing an FP equivalent (you are welcome to increment the vote tags):

* Example is too small (0)

* Example lacks enough "algorithmic complexity" (0)

* Relies on domain-specific libraries (0)

----
The example is boring, unchallenging

''I don't see how that is relevant. To some going to the moon is boring. All those space-ship parts to keep track of? Why bother.''

* Because is boring and unchallenging nobody should waste their time on it. There are other challenges that can help folks hone their programming skills. Life is too short to pay attention to all the crap out there.

* ''Hone your programming skills by solving practical problems instead of university lab toys.''

and badly specified. It takes quite an effort even to understand what it is that is required,

''I think if you study the screen-shots and some of the code, it should be fairly obvious. If you have an NT box, run it and play with it a bit. I will answer specific questions.''

* You see, therein lies the crap. Posting a challenge where somebody has to stare at your screens and study your "code" is crappy. Answer your own questions and write a decent specification of what the problem is. Not the solution, the problem.Plus it is completely idiotic to ask folks for the imbecility of software called MS Access that isn't even free. Common, real developers do not use MS Access, much less pay for it. Access is for secretaries.

* ''There is no decent open-source replacement for MS-Access. It might be crap, but crap is better than zero.''

and nothing valuable seems to entice the reader to solve such a grunt work exercise.

''You seem to be looking for MentalMasturbation instead of solving semi-realistic problems. '''If it's grunt work, then use your brilliance to automate it'''. That is how a real developer should approach "grunt work".''

* Automate the emulation of MS Access?? Why? Because Top posted an idiotic challenge?

* ''It only uses SQL from Access (assuming control tables are already filled in, but I won't hold you to that issue.) It is written in ASP (VB-Script), by the way. I am not promoting VB script here.''

Plus the "solution" as presented suffers from too bloody obvious SqlInjectionAttack which means that if the originator didn't put good enough effort to produce some credible code from where others can learn something, why should anybody else bother?

''I don't know what you mean. I cannot help it if you are an SQL-hater.''

* If you don't know what Sql injection attack you need to study. Google is your friend. As a subtle hint is when your app is exposed to an attack by the user (typically in Web apps but not only) whereas the user constructs a cleverly formatted input that will determine your dynamically constructed SQL to be other than what you expected, and pretty much whatever the user want, including malicious code. For example, the user puts a string terminator ', and a statement terminator ; in  an input field that you'll use to construct your query, and after that (which will terminate your SELECT ... WHERE) whatever he pleases like DELETE * FROM USERS. So the SQL the app will end up sending to the database will look like:
 Select ... where criteria=''; Delete * from users; ....

* ''Oh, so SqlStringsAndSecurity is what your yammering about. That is a side issue. It is an inTRAnet app and one can '''turn off all but reads''' for the data tables (in the DB connection setup config or as a special log-in). Stop looking for excuses.''

* Don't complain about SQL haters, because you expose yourself as a SQL ignoramus. No self-respecting SQL developer these days writes such code that is exposed to SQL injection.

* {"'''inn'''jection"? Isn't that what one does with a Motel-6 hooker :-}

These defects of the challenge have nothing to do with functional vs OO vs visual basic or whatever, but it's just not enough of a challenge. Opposing it to FP as if such a challenge can prove anything one way or the other about the merits or demerits of FunctionalProgramming is completely misguided.

''The only valid criticism is unclear documentation. I will answer specific questions if you first bother to study what is available.''

* Why should anybody bother to install a crappy database for secretaries that costs tons of dollars? Let's get serious, Top.

* ''ArgumentFromIntimidation again? It ain't making you look good. You sound more like a playground bully than an articulate intellectual.''

----
[On the contrary, I'm finding just installing the software to be enough of a challenge. 3 days and counting just to get GHC, PostgresQl?, WASH, c2hs, and Dbconnect installed. Ugh.

Any idea why ld is giving me "cannot find -lpq" when I try to link? libpq.so.3 is in /usr/lib and I've got a -L/usr/lib option specified. -- JonathanTang]

Like I told you, unnecessary wasted effort. What do you hope to learn in the end? That Haskell is entirely inadequate  to tackle database driven web apps. I could have told you long time ago. How much time do you think SimonPeytonJones spends on integration with ODBC issues? You just hope that a bunch of developers out there programming a couple of afternoons now and then are going to offer you a software stack comparable with that which is the results of many man years dedicated to the comparable .NET or Java solutions? The only sane solutions to such difficulties is to move over. Life's just too short to be wasted on lost causes.

It is wishful thinking to think you'll get lucky with Haskell. After you'll fix the libraries, you'll discover that WASH will not hold to any kind of stress test, or that you'll have problems adding SSL, and so on, so forth which would make the  idea to use such a stack for any kind of production software completely ludicrous.

''Relax. It is a toy example. It just has to run for a few local tests. He's not planning on taking on eBay. I would think that there is something more nimble than PostGre though. -- top''

On the contrary, I expect to find that Haskell is quite good at CrudScreen''''''s. I'm looking for at least a factor-of-2 improvement in line count; that'd be about 200 lines max. I was talking to DougMerritt a couple of days ago and he also didn't see any reason why it wouldn't work. He also mentioned that a friend of his had read ArrayDeletionExample, tried a couple of the techniques in a custom biz app for a Fortune-1000 company, and found them quite helpful.

SimonPeytonJones doesn't spend his time writing ODBC drivers, but that doesn't mean other people don't. And the Haskell community is small enough and elitist enough that they often turn out very nice designs. I'm quite impressed by WASH and HaskellDb, for example, even if I'm using Dbconnect instead of HaskellDb in the end.

I'm using PostGres because it's the only supported DB for Dbconnect. My options, living in the open-source Linux world, are PostGres and MySql. Dbconnect (which comes with WASH) doesn't work with MySql, so I'd be forced to use HaskellDb or some other database library. HaskellDb has its own dependency stack (for example, it requires HSQL) and I really didn't want to go through dependency hell again.

Also, my own language design efforts have recently shifted towards the web/database apps world (as that's where I think existing solutions suck the most), so this is good practice with various database webapp solutions. I'm already familiar with MySql; add PostGres and I've got both major OpenSource databases covered. And WASH is state-of-the-art as far as web frameworks go. I want experience with both before I try and write something better. -- JonathanTang

----
Top... just one question. Did you have a hand in designing Challenge #6 here?

''Yes, why? The general concept as a comparison was brought up by John Urberg IIRC.''

That explains it. The article raises some objections to OO that I've heard you raise, and I was curious if I was right in reading your hand into it.

It's pretty typical stuff for top. Demand vaguely, then rant and rave when people fail to deliver according to some exacting specification that's super-secret. When presented with answers, he selectively filters out anything he doesn't want to hear.

''Damn, you make me sound so evil. What specifically have I "filtered" out?''

He hasn't even managed to write a coherent specification for this "Challenge Six".

''All flames aside, I do agree that Challenge #6 is underspecified as it stands. Every time I consider trying to do it, I have to scroll up and down repeated trying to match requirements, terminology and implied data from screenshots. A clear, concise summary would be better.''

If you have to scroll back and forth to the same areas, then open up two browser sessions. Or, print them out on paper and spread them on the coffee table. I often do that when studying poorly-documented systems. I will try to provide some more details {since added below}.

''Or even better, maybe we should have a short programming contest. Challenge #6 '''is''' boring, but that doesn't mean we couldn't make a slightly more fun project and then everyone could submit and judge results.''

I don't know why you guys find it boring. CrudScreen frameworks are generally interesting in my opinion. They are not trivial because of all the potential options and variations they can have. It is tough to make a truly generic one that handles all possible business requirements and it still be strait-forward. But it makes for an interesting challenge to search for the best mix of such competing factors. (This example is not meant to cover all possible combinations and features by any stretch. It is pretty much an illustrative experiment.) -- top

----
PageAnchor: more_details

'''More Details'''

Let's take a look at this screen-shot:

http://www.geocities.com/tablizer/rptscrn1.gif

It is using report number 2 (selected from the simple RptList.asp report listing). The listing (output) at the top is based on a query created in part using the "Reports" table. If you look at the "selectClause" column in the "Reports" table screen-shot (see original page), you will see an asterisk. This means the SQL Select clause selected every column in the table. In this case the data table has 6 columns. The database table is determined by "fromClause" column in the Reports table.

The top part shows the results and the bottom part shows the user's criteria because the criteria is repeated (echoed) to the result page so that the user can modify their already-entered criteria if they want. This is why both are on the same page.

I'll come back to the results later. For now let's focus on the bottom part, the criteria input form. This is generated using the "reportCriteria" DataDictionary, illustrated in part below:

http://www.geocities.com/tablizer/dict2.gif

These criteria fields come from the rows that have "reportRef" equal to 2. See the second column. (The column name is partly truncated to make room, but all the columns are shown in full in the schema on the original page.) You can see that the rows with reportRef = 2 match closely with the above screen-shot. This is a very typical DataDictionary. "Field-dictionary" would probably be a more fitting name in this case because these do not necessarily correspond one-to-one with actual database columns.

The only real oddity in this set is the price range (lowest price to highest price). Our data dictionary does not have a native "range" type, so we used two fields that refer to the same database column (listPrice). When somebody puts a value in a criteria field, by default for numbers it generates an SQL sub-string such as:

  and X = 999
This assumes the database column is 'X' and the user entered '999'. It basically just uses the WhereAndAnd technique. The "fmtType" column determines whether it has quotes around it or not. And, for strings the default is a "LIKE" operator.

But for ranges we don't want an equal comparison. Thus, the "Comparer" column has ">=" for the first range part, and "<=" for the second. If the user types in "5" and "20" for the range, both "listPrice" rows will generate two strings:

  1. " and listPrice >= 5"
  2. " and listPrice <= 20"
These are then internally concatenated together as part of the final SQL statement. (A fancier version could perhaps use an SQL "BETWEEN" clause instead.) Here is the snippet of code that performs most of this process:

	'---Comparer
	if isblank(rs("comparer")) then
	useComparer = " = "
	if fmtType = "T" then
	  useComparer = " LIKE "
	end if
	else
	useComparer = space(1) & rs("comparer") & space(1)
	end if
It is basically just constructing "AND" sub-clauses for the final SQL, as described in QueryByExample and WhereAndAnd.

The two other data dictionary columns that play a significant role in the range approach is the "keepWithPrior" column and the "sequence" column. If "keepWithPrior" is set to "true", then the input field stays on the same line. This is just a formatting nicety issue. What is "prior" is defined by the values in the "sequence" column.

The "in stock" option is format type "Y", which produces a pull-down list of "Yes", "No", and "(either)" as the choices. One can also use the "theList" column to enter a comma-delimited list of options. This particular example does not use that feature. A fancier version perhaps could separate the list description and the list value. (Note that I called the column "theList" instead of "List" to avoid overlapping with reserved words. I don't know if it would be an issue in this language or DB, but I generally avoid column names that risk such overlaps.) -- top

This is still completely unhelpful. You're explaining "how" and not "what". A good spec states the business requirements, and only the business requirements. It should have no code (unless one of the business requirements is that managers can write code for it), and it should lay out everything that a user can do with the system. -- JonathanTang

{That is not true. Only a little bit talks about code. I am mostly describing the UI and how it relates to the control tables (which I suppose is a kind of UI for app admin people). You don't have to use control tables if you don't want. You can use EssExpressions if you prefer. I won't complain if it is anything declarative (non-executable) and fairly easy to edit. Personally I would rather use a table editor for entering such info, but if you dig EssExpressions for whatever reason, go for it. If you want it in requirements format, then try this: "Must provide a way for an experienced operator or administrator to add and edit report configuration information without having to know programming." -- top}

''Top, it '''is''' true. The document is awfully light on requirements and awfully heavy on how something should be done. We're well aware we don't have to use control tables. What we're confused about is the exact specs of this boring system. Tell us what we need to store, what we need to be able to query upon, the limits of those queries, the specifics of the preview function, and all the details in between. You keep telling us the how like it matters or gives us insight. Like I said before, I had to keep referring to the pictures just to figure out what data would be operated on. That's not acceptable as a requirement professionally or personally.''

''The "how" is especially useless when you're asking people to do the "what" in a better way. Could this be a case of GoldenHammerTintedGlasses? :)''

Ask questions. I added a section below for that with an example. And please stop mentioning that you find it a boring problem. Unless you can tie boredom to being less practical as an example, there is no reason to keep bringing it up. The people who sign our paychecks are not necessarily in business to entertain us. If it bores you that much, just admit FP defeat and move on. If you can find an "exciting" somewhat practical and typical biz example, you are welcome to present it as an alternative. -- top

''The problem is that you don't sign any paycheck, and as long as you offer it as a challenge, you have to assume the responsibility to make it interesting. Otherwise you have no reason to complain that people criticize it as boring and waste of time.''

  Boring != Waste_of_Time
Or to be more precise:

  Equiv(Boring, Waste_of_Time) != True
You have failed to find a better example for biz apps, so you are stuck with mine. One thing I have learned from the biz world is that you better have a ready alternative before criticizing the existing state.

''Let me break down the math for you, Top:''
 (Boring * No Money) ^ Trying to educate someone who rarely listens + Student likes to twist examples so he can ignore them 
	= Waste of Time
''We're doing this for you, not because we have some deep-seated desire to implement mind-numbingly boring CrudScreen apps. This kind of work is why we have software interns. Me, I'd like to write something that, you know... '''does''' something, besides tell a database what to keep.''

Do it because you want the geek bragging rights of using FP to kick a skeptic's ass in the biz domain.

Sounds like sour grapes from somebody who is stumped to deliver their claim. As far as "rarely listens", I don't want words. They only lead to LaynesLaw battles. I want to see the claims in code, not moving lips. By the way, Phd's in India are only $2 an hour on the world market. Who needs interns? -- top

Ahem, I'm a software intern, and the work I did this year is ''much'' more interesting than this. This reminds me of the stuff I did freshman year (I didn't go back to that company, BTW), or what I the stuff I still do for free for friends. Except, I'm not a freshman, and we're not ''that'' friendly. ;-) BTW, PhD's - whether in India or the U.S. - are a lot more than $2 an hour. There're plenty of good programmers in India. But the good ones get paid as much as the good ones in the U.S. -- JonathanTang

''I already proved my claim. See the above equation to see my feelings. We showed you examples, you turned them down. I'm not an FP golden hammer addict. I have lots of tools in my toolbox, including most of yours. And no, Top. There are no PhDs from respected universities making $2/hr for software in India (barring freak examples). This kind of business app has no challenges. It's just like stamping out license plates. It's good for people who can't handle more than mindless repetition. I might do this kind of work to get a product out the door (because I'm being paid, and someone has to do it) or because I'm the FNG of the team, but not because I'd want to. It's like taking pride in making cookies in an EZ-bake oven. Sure it's great, I guess, if you're someone who finds that kind of thing challenging.''

{You are confusing personal entertainment with use. I too would rather be doing AI research, but there is no money in that except for the lucky few. If CrudScreen stuff were as repetitious as you say it is, then it would be easy to automate and you could put hundreds of thousands of programmers out of business. As explained in DomainPissingMatch, biz apps have plenty of complexity.}

''This application '''has no use.''' While some people may make an honest living doing such work (using tools which help automate it and expedite development, of course), they receive a paycheck for the work. We will not.''

----
I am not sure what you (realistically) want then. '''I just want to see an FP size demonstration with code that is applicable to my domain.''' That is all the hell I am asking for.

''Here's what I don't get, Top. If you really want to see it, then why not try it yourself? It's not a matter of time, because if you had just started coding you'd have spent a hell of a lot less time than arguing with us for hours on end. Pick a language that has something similar to HigherOrderFunction''''''s and go to it (either build your own, or download and understand some of the example code that exists). You can pick all kinds of different languages for this. At the very worst, you don't like it but you understand it much better. At best, you like it and you have another tool to use. Both options are wins for a software developer.''

There is plenty of other more promising stuff to study. I don't see enough initial value in FP to pursue that path right now. I just wanted some realistic evidence from those who claimed "significantly less code". I thought that one would be easier to demonstrate since it is more easy to measure than the vaguer claims such as "cleaner code", etc. -- top

''No. What you thought was that you could get one of us to do the groundwork for you. Top, I don't care what you study, or what other "more promising things" you study. You do what you like. But stop saying, "You can't prove it." We don't have to prove anything to you, Top.''

There are others who also think FP fans are mostly full of hot air. ''(Implying that there is a wider audience for proofs.)''

* ''Name a few people who say functional programming is mostly hot air.''
* They are mostly unsigned. As I come across them, I will list the references here.
** Under GoldenHammerPoll, somebody named closures and HOF as their GoldenHammer.

''When we asked you to prove that TableOrientedProgramming was superior to ObjectOrientedProgramming (as you claim), you didn't respond with proof. You responded with talk. Now you demand code from us.''

I don't claim it to be objectively better (unless perhaps consistency can be measured). Just not objectively worse. '''You made an objectively-measurable claim and I am simply holding you to it.''' Are you afraid of the RealityPolice?
* '''I''' didn't make any such claim. I claimed that HigherOrderFunction''''''s were a GoodThing, and could decrease code size. I didn't say they always would. All I said was that they're very useful in certain common tasks.
* ''Somebody made such claims. Maybe they cowarded away. I never disagreed that they offer incremental benefits for some things. But the claimers went beyond "incremental".''

''Here's my proof, and the way '''I''' decide what to study next. I see something as I read around online that is not just interesting, but was used to make an amazing product. I study how that product was made. There are tons of bang-up awesome things written in FunctionalProgrammingLanguage''''''s and in ObjectOrientedProgrammingLanguage''''''s. These is absolutely nothing interesting out there written by you.''

When you get to define "interesting". ''Top, maybe we should take a poll on the wiki. But even worse, there is nothing out there by you at all! For all we know, you've never written more than tiny demo apps.''
* I am looking for practical, not necessarily "interesting". Those may be orthogonal. What do you mean by "out there"? I work on custom biz apps. Being custom, they are not "out there" by design.

''Get someone else to do your homework, Top. I can tell you that I don't have time for your kind of BS without somehow eroding my position and claims. This is not college, Top. Eventually you stop getting examples from people and start getting scenarios and patterns. It's up to you to make the examples that will satisfy you.''

I don't understand Top's attitude either...

It's not my or anyone else on C2's responsibility to educate you, Top. I gain ''nothing'' from showing you FP. Heck, if you really cared enough to learn it, I could end up gaining a competitor, which would be a net loss for me.

I learned Lisp because a couple folks (PaulGraham, PeterNorvig) made a couple tens of millions of dollars with it.

''Bill Gates made much much more off of assembler (DOS) and Visual Basic (written in C). Do you really want to measure by money made? Hell, COBOL would probably win, followed C.''

That seemed like a good reason to me, though perhaps I have different values than others. I learned Haskell because PaulGraham mentioned this whole BlubParadox thing, and I realized that his and other SmugLispWeenies' comments about Haskell made Lisp seem very Blubbish. It also helped that GuySteele, who literally wrote the book on Lisp, is quite a fan of Haskell.

I'm interested in this challenge because I think actually programming a real application in Haskell will make me a better programmer. I'm interested in putting the results up because I think some lurker (probably not TopMind, I've basically given up on him) might find them interesting. I would never have learned about advanced programming languages had others not put what they know up on the Internet, free for the taking. I figure here I can give back in the same way.

But I've been spending the bulk of my time wrestling with PostGres and libreadline and the fact that Mandrake can't put together a decent RPM to save their life, which is not educational for me at all. And I doubt anyone here cares what a Haskell webapp looks like. So if you really think that you have better things to learn, and no anonymous lurker speaks up, I'll go do more productive things instead of responding to this page. -- JonathanTang

----
''And like I said, we're not doing this because we stand to gain anything. We're doing it because we're trying to explain something very simple to you and you're stubbornly saying, "No no no."''

Nope, it is "Show me, show me, show me". That is '''not''' the same as "no". I am from the MentalStateOfMissouri. Why are you painting me as evil or stubborn just because I won't take your word for it???

''I won't make any DomainPissingMatch claims, and I have not done so to-date. I leave that to the Haskell and Lisp gurus. Sometimes, a functional approach to programming can save huge amounts of code (or give other favorable results, like ease-of-extension or maintainability), like in my weather example. Likewise, sometimes it does not. It is the same with OOP, COP, and LOP.''

I can't see the real code and/or requirements for the weather example. I provided runnable code and offered to answer any questions. You have not reciprocated. I think you are possibly exaggerating with the weather example. I don't want to have to take anybody's word for it. I want to see the code land on the figurative Whitehouse lawn.

''If you're really going to maintain that this is not true, then this discussion is clearly over, because it's literally an argument between someone who refuses to use anything but a hammer (And your excuse about your job could be, "But all I have to do is hit nails.") and someone with a toolbelt saying, "Look, when you work with a screw, a hammer really sucks!"''

'''''I suggest we try to form an application idea and hold a little code-off. It might even be fun. We could implement something like blogging software and see who would come out on top. As long as we will actually use in real life, and keep the scope small, we might actually get some real valuable information (and some working pieces of software, besides).'''''

Like maybe implementing TopsQueryLanguage :-)

'''I think it's time to concede. Most functional languages cannot implement CRUD screens with less code than FoxPro, a tool designed for writing CRUD screens. Top, go take a victory lap. Then go write a thermal flow simulator in FoxPro.'''

It sounds like a thinly disguised ParadigmPissingMatch insult, but I will take it as a victory. Actually, FoxPro was/is not very good at CrudScreen GUI's. I like it for its nimble table handling and CollectionOrientedProgramming features, not for it's GUI's. And, maybe FoxPro does indeed suck at thermal flow simulators. No tool is best for everything. I am curious, though: Do engineers write flow simulators from scratch? It seems like a problem that is standard enough that existing tools or libraries would already exist for it. -- top

----
'''Q & A'''

* Q: What does "theList" column do?
* A: Results in a pull-down list of the given options. The list is comma-delimited and assume that leading and trailing spaces are trimmed.

* Q: Are the comparisons case-insensitive?
* A: I assumed case-insensitivity, but didn't really enforce it, so let's make that a non-issue (you won't be docked if case issues arise).

* Q: Must we use SQL as the report specification language?
* A: It may make it harder to compare if you don't, but it would probably be unfair to rule it out. In practice it would be easier to find an "operator" who knows SQL.

* ...

----
(Moved from ObjectiveEvidenceAgainstTopDiscussion)

''It doesn't matter if SQL is implemented by Higher Order Function; The SQL IS language WITH limited form of Higher Order Function.''
''Your SQL query statement is so compact and comprehensive because:''
*''you are allow to pass a function that tests if an element should be include in the search result (WHERE clause).''
*''you are allow to pass a function that filter the result of query to get only desired column (SELECT clause).''

''SQL is actually like this (#' denotes function that can be passed to another function):''

 filter-for-interested-column( #'select-clause, remove-if-not(#'where-clause, join-table("table1", "table2",...)))
 
''you would not have this SQL power if SQL didn't have HOF in its language. If SQL is like C/C++ you will be doing this in SQL:'' 
 //hypothesis Non-HOF SQL syntax
 join_result = join("table1", "table2",...)
 filtered_result = []
 for( record in join_result){ //can't have where-clause because I cannot pass a function into another function
	if(is_valid_by_where_clause(record)){
	filtered_result << record;
	}
 }
 result = []
 for( record in filtered_result){ //can't have select-clause because I cannot pass a function into another function
	result << 
 }
''You enjoy so much of this expressive power in Data Retrieval Domain Because of HOF. Yet you said HOF is not much useful in your domain, YOU ARE USING IT YOURSELF ALL THE TIME AND YOU CAN'T LIVE WITHOUT IT. Now, In challenge #6 you raise, NO FP language is going to reduce the query code to be smaller than SQL; Because FP with HOF and SQL solutions WILL BE THE SAME, It comes down to as much the same size. But NO language without HOF, is going to be at the same size with FP or SQL. That's why we said HOF is so useful, after all you don't do just select/insert/update/delete. We don't other operation too!! See after you have query all the data out to series of row, with your procedural language, you have to come back to for loop again. But No, FP'ers don't have to do that. They can still enjoy then same expressiveness as in SQL with whatever the operation they are going to use!! They can pass criteria, filtering to everything, not only SQL engine!!! That's where the code of FP language will start to shrink from Your Procedural parts. But, there are so small things to do in that challenge after the data is extracted, people didn't bother to accept the challenge because it wouldn't be anything left to challenge about (You use HOF feature to reduce code size already, and you are challenging FP to use HOF to reduce the code size again; how is that possible?). (Don't bring eval into here, that's another topic).''

''Why don't you want the same level of expressiveness as in your query? You are happy that you have SQL which itself is HOF enabled language, yet you are all against HOF? What is that?''

This may be a definition issue about what HigherOrderFunctions really are. Your definition seems to be loose enough that the "eval()" techniques I mention may also be considered HOF. Your conjecture is an interesting approach, but I still would like to see an actual example of it reducing code in a biz app. I selected challenge #6 because it is far more representative than the other examples given (not perfect rep in absolute terms, though). But, then you suggest that something about it makes it not very reducible. Let's call it Factor R. What we need to find out is whether this Factor R is because it is closer to a real biz app, or something else altogether. I have found an example of something that is not very reducible by your techniques it seems, so it makes an interesting exploration tool for FP relevancy, would you agree? It could be that SQL already absorbs the primary benefits. SQL and Eval together seem to cover so much of the territory that FP would otherwise cover, that there is not a lot left over for "direct" FP to help with. -- top

''SQL and eval together *are* FP. Especially when you're storing operations in tables.''

Again, I think that is a stretch of the definition of FP. Somebody else called expressions in tables OOP. I see it as DataAndCodeAreTheSameThing. My ideal is exposing the run-time engine (AdvantagesOfExposingRunTimeEngine) so that it does not matter, each is just a view of the same thing. I suppose what one calls it depends upon their favored EverythingIsa view. Eval does not really care what is passed to it, it just evaluates it. It could be a function, expression, etc. Whatever they are called, I find them useful. The real issue is whether going to a more "native" or full-blown FP will make a significant difference.

''I find it hard to show the benefit of HOF to you because you don't even yet know the different between HOF and Eval. Eval can do everything HOF can,''

* You can't store HOF in databases (as text). It is somewhat similar to the "object identify" fights that rage from time-to-time.
** ''You can store the method/function 's name as string. And use reflection to get the method to call, no Eval needed here. I supposed you only put the method name, not the whole implementation, in DB, correct? Cause I don't think that's good practice. You can raise the point that not every language has reflection, but not every language has Eval also.''

''But it's not necessary to use Eval. You can wrap your whole Program in one eval() too, or you can use Eval instead of direct assignment statement also, but that would be stupid. This is the same, you can mostly use Eval inplace where HOF is used, But Eval has it's own quirk that it should not be used as much as HOF. That's why we prefer HOF over Eval.''

* Again, I don't need to use eval that often. I described why already at my personal page.

You don't use it, fine. We don't know what kind of applications you write either. Obviously for PutTheDamnDataOnTheDamnScreen kinds of scenarios one doesn't need much advanced techniques. However, you're trolling when you generalize from such a limited experience and such a limited education.

''It is not me who generalized. You are guys are the ones implying that examples from domain A are applicable to domain B.''

* Actually, you have to give up on this bullshit. Or otherwise please provide precisely here, which one of "us guys" and precisely where, did what you dishonestly accuse us of doing. Nobody that I am aware of provided anywhere in this endless trolls an implicit or explicit claim that FP techniques generalize to every domain. This is the StrawMan you invented yourself without any justification whatsoever.

* ''Complaints of confused identity coming from somebody who does not sign? That's sad but funny. As far as the second part, there have been multiple places where somebody suggests, "if you were smart enough you would be able to extrapolate our examples into your domain". Did I hallucinate that also?''

Running a quick statistic over my current Java project, out of 1028 classes some 588 classes were  inner classes, thus some form of closures (Java's more verbose corresponding concept to HOF). Of course, this project is peculiar because it is multithreaded and depends on all kinds of events happening between components distributed over the network in unpredictable (non-deterministic) fashion so the use of closures was dictated by a framework needed to keep the concurrency in check. But other Java projects that you can examine (open source) have a healthy doze of closures as well - and you cannot complain that java projects are as peculiar as FP examples coming mostly from university. Thus people do use closures and HOFs, and for you to pretend that you know better that constructed strings can substitute those, well, that is pure unadulterated trolling. It's again a form of ShiftingTheBurdenOfProof. -- CostinCozianu

''Even if it was true that they were common (we only have anecdotal evidence here), that does not necessarily imply they are better.''

* No, but it knocks down your silly argument that one needs them so rarely that for the very few occasions when they are needed one could incur with the drawback of eval.
* ''Well, these anecdote fights are not getting us anywhere.''
* And that's precisely where you want to go. That's why you spread your ignorancy, reply to every reply just have more ThreadMess, even when your lack of knowledge is so rampant that an honest person would abstain from commenting at all, everything with the only bottom line that thanks to your trollish ThreadMess the wiki page leads nowhere.
* ''If I am so dumb, why can't you produce "significantly shorter" code than what an alleged dummy produces? Put your code where your mouth is. Insults are cheap. This is a demonstration topic, not a wiki peer opinion survey.''

The main reason why dynamically constructed strings '''should not be used''' as a replacement for functional values (HOF, closures, inner classes in Java, etc.) is because when you construct a string and eval'ed later the programmer has a much harder time proving (convincing himself to a degree of certainty) the correctness of the execution of constructed string. For example, one would have to account that any values pasted into that string from current variables don't interfere with the syntax of the language, and that's just one of the many crappy details that become a burden when one goes the eval route for all but the simplest of examples. Remember programmers live with a ProofObligation.

''This sounds like it follows a typical debate between static typing and dynamic typing. No need to repeat that stuff here.''
* No, you're trolling to get rid of arguments you cannot respond to. Where do you see types brought in the above? Even in a language with dynamic typing it is much easier to '''reason''' about a code that you actually '''see''' and conforms to the syntax and semantics of that language, rather than to reason about the code that you don't see but it will be computed sometime in the future during the runtime phase. That's very basic.
* ''In practice, eval's have not been a big source of problems and errors for me.''
* But you haven't used them much, either, by your own admission. So you're in no position to tell whether they are or they are not a problem when used instead of closures. And by the way, you have not used seriously any language with closures either, so you're in no position to make a comparison either. All you do is argument from ignorance and wasting everybody's time.
* ''Isn't the purpose of this topic to explore a specific example and demonstrate claims with counter code rather than brag about each other's own experience? If the example is not capable of demonstrating FP strengths, it seems logical to identify and come to a consensus on exactly why. The domain library issue is insufficiently explored, as described earlier.''

As a convincing proof that you don't know what you're talking about, the very code you presented for ChallengeSixVersusFpDiscussion '''is not correct''' for all values provided via user input (you construct SQL strings to be submitted to the database). What you should have used there (had you known any better) was prepared statements with bound variables, which is a mechanism somewhat similar to a closure. And this closes the "strings versus closures" discussion.

''One has to check the SQL-targeted values either way. Unless you reinvent and mirror SQL locally, the computer is going to just treat it as a string even in the tightest of languages. (I talked about SQL and security earlier above.)''

* Go educate yourself, Top. You don't know even basic stuff about databases other than FoxPro or Access. One needs only use '''prepared statements''', and the DBMS provided driver will take care of the safe binding of values to parameters of the queries. No need to handcraft your imperfect code that will maybe do half the job that the ODBC driver can do for you.
* ''You are correct. Stored procedures greatly reduce the change of that happening. I failed to consider them here. However, in practice I find them a hindrance to productivity and generally only recommend them on bottleneck areas or if security trumps productivity (client decides, not me). Plus they are not as dynamic: you can't arbitrarily add conditional clauses, for example, without lots of riggamoroll. Example need: QueryByExample, which this challenge kind of implements. I already wrote something on that around wiki. I'll try to find it if you are interested.''
* Please stop jumping in with idiotic responses when you don't know shit, Top. Go learn, take a BookStop for a change, it'll do you a lot of good. I wasn't talking about stored procedure, I was talking about '''prepared statements''' which are different animals that only major ignoramusses that pretend to be expert in database application can confuse with stored procedures. Prepared statements are a mechanism that if you did use in your poorly written code you could have spared yourself the embarrassment of bragging about code that's full of holes, when it would have been trivial to make is safe. Now go take a break, do some googling and '''start reading for a change'''.
* ''"Prepared Statements" appear to be a Java-specific technology. I am not a Java programmer. And, they don't handle arbitrary clause creation that I can tell. They plug values into pre-existing slots. That would not work for QBE.''
** PreparedStatement''''''s are part of the SQL'92 ANSI standard. They are in no way "Java-specific". They are the only statements that accept bind variables. The fact that you don't know about them reflects poorly on your experience and skill set.
* '''Both claims are false.''' You do look like a lousy learner, but you're quick to come back and respond one more time from ignorance, nevertheless. Go back to study, TOP, and don't come back any time soon, cause you're wasting people time for nothing and create lots of ThreadMess for no good reason. Fix your ignorance and come back later. And, by the way, do not hurry. You have a lot of fixing to do.
* ''I checked 4 books I have on languages and databases, and none of them mention it. Also, if you Google for the term, about 90% of them are Java links. This implies that most languages and API's ignore them accept for Java. A lot of stuff is in the SQL standard that is ignored by the industry. How else do you explain the 90%? How is this relevant to #6 and FP anyhow? If you want to add prepared statements to your FP version, be my guest. This appears to me to be a red herring ploy. It seems you work much harder to try to embarrass me than show FP in action.''
** Google says there are about 38,900 pages with "SQL" and "prepared statement" and only 21,400 of those include the word "java". That's about 55%, not 90%. Here's some excellent advice from Oracle about prepared statements: http://asktom.oracle.com/pls/ask/f?p=4950:8:::::F4950_P8_DISPLAYID:1993620575194. The gist of that page is: "This is a fact, a rule, a law -- use prepared statements EVERYWHERE. Use STATEMENTS almost nowhere." In my experience, this is common knowledge throughout "the industry".
** ''My small survey of hits kept leading to Java, even if the initial link didn't mention a language. As far as whether they are the magic solution to SQL hacking, let's move that to SqlPreparedStatements.''

Buy better books. Oracle and Sql Server, the two biggest vendors, and others I'm sure have supported prepared statements for a while, doing dynamic sql without using prepared statements is ignorant and unsafe, and amateurish at best. Prepared statements have nothing to do with java and counting google links while pulling statistics out of your ass isn't sufficient research to rebut something you're ignorant of. You're lack of knowledge on all this is astounding, as usual, amazing someone pays you.

[EditHint: move sql-injection stuff to SqlInjection or SqlStringsAndSecurity]

----
1. Go see [Marker 00001] in DynamicStringsVsFunctional That explain why HOF is more suitable in most case than Eval. And
2. I don't know why such expert in Biz app and DB like you don't even know about prepared statement. That's funny.

''I have worked in about dozen companies as both an employee and a contractor, big and small, and not a single one used prepared statements, at least not in volume. A good many used stored procedures, which are very popular. Why I haven't encountered them, I don't know.''

* Because you are an ignoramus, that's why. Because you rarely pause to study, but you troll endlessly about your strongly held convictions. With such an attitude it can hardly be surprising that you don't know tons of things.

''Out of a hundred technologies, to pick a sample size, if one works for a dozen companies and each one is used in 50% of the companies, then by typical Piosson (sp?) distribution, a handful will probably slip through. But it is immaterial to this topic. (By the way, I have not seen how they allegedly allow dynamic clause creation. Anybody have a link? Thanks)''

''Further, a Wiki '''Double Standard''' has been exposed. Somebody suggests using nested loops instead of joins, which is considered poor form in the industry, and none of you say a single word about it. Yet you find that I don't know about an obscure database feature outside of Java, and suddenly I am stamped the dumbest guy on the planet. You have been caught double handed. Your bias drips with the vital fluid of twisted truths and social manipulation.''

[I agree with TopMind that the attack seemed unwarranted. I hadn't heard of prepared statements by that name at all. They're mentioned only very briefly in Date & Darwen's GuideToTheSqlStandard, third edition, as "preparable statements". I'm willing to believe that's a failing on my part, but the vitriol over that detail doesn't seem warranted. Perhaps they go under different names in vendor's implementations. -- DanMuller, DeleteWhenCooked]

TOP's particular ignorance of this basic mechanism is not the issue. If TOP was just a mere ignoramus he'd be excused and spared the public embarrassment. However the difference is that Top's '''ignorant and proud of it'''. His self-infatuation in this case and other cases is beyond belief.

I told him of the problem a few weeks ago and told him it was trivial to fix the code against "SQL injection attack". A simple google for it  would have led him to prepared statements, he instead handwaved about it. Above I told him once more. After he was told he was wrong and that he should have used "prepared statements with bound parameters" any self-respecting contributor would have checked first what that was. There are countless articles on how prepared statements save yourself the embarrassment of SQL injection problems. He instead posted '''four''' more irrelevant replies without even thinking about it, just hoping that by mere luck he may find something that can stick. '''All''' his replies above cannot be justified for a contributor who wants to participate '''honestly''' in a discussion.

This trolling attitude is '''uniquely branded ''' to TopMind. I don't know of any other contributor to Wiki that after given a clear indication that he's wrong and why he's wrong he'd contribute more irrelevant replies. The sole purpose of such ThreadMess''''''es is solely for TopMind to titillate his own ego.

Prepared statements are a well-known mechanism. In ODBC the API name is SQLPrepare, in ADO if I'm not mistaken is Prepare, and tons of examples are everywhere one cares to read about how to interact with SQL databases. Almost all vendor specific client APIs had a prepare call for longer than I can remember. frankly, I cannot even remember when the Prepare call was first introduced in a client API. For the duration of my software engineering career it was always there in one form or another. If you think you have a solid knowledge of things database related but you are not familiar with this concept all I can tell is that you have a serious problem with the selection of books or documentation material for your study, but that's a side issue.

How is this particular example relevant? Because prepared statement solve a problem that also happens with eval: the interference between values picked from environment (especially string values) and the language syntax that the evaled string has to conform to. Because of these, to everybody but to TopMind, it is perfectly clear that thinking about the correctness of the evaluation of a dynamically composed string, is much harder than thinking about the correctness of a closure. He repeated by the way of handwaving that such things are not important and when I pointed out to him that such an interference bytes his ass in the very example he posted in a domain that is supposed to be his area of expertise, he ended up generating this ThreadMess rather than taking the opportunity to learn.

Because above everything else, his positions need defended at no matter what cost. Honestly or dishonestly, informed or ignorant, these criteria are totally outside Top's concern.

''Sorry, but I still don't see the connection to this topic. Eval was not an issue in any of the SQL protection comparisons. It is a toy example and intended for intranet use by a handful of managers who would have read-only access to most of the tables anyhow. And, there are multiple approaches to protecting the SQL, each of which I feel is not relevant to this topic. Further, I am not sure standard ASP (a now dead language) supported them and maybe didn't support the clause-addable version, which I suspect does not even exist. -- top''

Another mindless reply (5th). Why don't you go and read up a little bit, including what was written above and when you are able to do more than to argue from ignorance come back then.

''Sorry, but I missed the direct connection to the topic. You seem preoccupied with compile-time checking and generally anti-dynamic. We don't need to repeat that debate here. '''If you want to use PS's in your version of #6, be my guest'''. That is why it does not matter. -- top''

EditHint: move the prepared statement stuff to SqlPreparedStatements

Here, sample in javascript, with some of your vbscript as the body, to add a little realism. You're proved wrong top, now extrapolate a little and see how this will help you.

 //with each row, a higher order function that would eliminate 6 repeated loops within your sample
 //and guarantee never having a bug caused by forgetting to movenext, or close.
 //this is a function you desperately need
 function withEachRow(sqlString, aFunction){
	var rs = stdConn.execute(sqlString);
	try {
	while(!rs.eof){
	  aFunction(rs);
	  rs.movenext();
	}
	} finally {
	rs.close(); // ohhh, it's better than your code, it'll close the connection, even under an error condition, yours won't.
	}
 }

''For one, we might not want to close it after each QueryAndLoop but leave it open for several loops to gain a little efficiency. Second, good web systems allow global default handlers to be defined for database errors that can close the connection, and even better the connection should automatically close after each HTTP event (submit). There is no reason to keep it open, error or not. Third, sometimes we want different handling for zero or multiple results. For example, a custom "sorry, not found" message for zero result rows, and sometimes there should be one and only one result, such as a count query. Yours is not generic enough to handle all that and making it generic enough will just bloat it up with creeping featuritis, especially in a larger app. Finally, a different contractor would come in and have to learn your custom query looper and all its little features, cranking up the learning curve for shop-specific conventions. I agree that it may shorten some loops by about 1.5 LOC (depending on language), but may result in only a few percent total. The claim was "significant reduction". 5% is not even close to what I consider "significant" to mean. -- top''

So this one trick reduce 5% of the code. not enough to claim significant reduction. But who say we can only use one trick per app? Bring in another loop HOF construct, reduce another 5%. Bring in some MAP, INJECT. reduce another 10%. and all that it reduced also increase stability in your code; every part that use "withEachRow" will never leave its connection open. hand typing may forget one. reduce 15% of code size and increase stability of code by 15-30% is not something you can do easily. and that's significant because it BOTH reduce code size and increase stability of application.

For the record, I don't see why it would make "withEachRow" bloated just to add <close_on_exit?> flag and <if_empty_rersult_do> function callback, both default to current behavior; it will only add three more lines to above code. And you are being funny here about "sometimes query only return one result". Duh, the above code is utility for use in LOOPING over ROWS of result set; if there is only one row then DON'T USE IT. It's funny you are complaining like "'for loop' is not generic enough because in some case we will only want to execute only one time"; Don't use the for loop there, man. For query that return only one value like <count>, you already know that it will never return more than one value, so someone would be so stupid to not understand that above code is not for such usage. And The above code already always close the connection, error or not.

 //so you can keep writing your nested conditional code, within the safety of a nice HOF to protect you
 //and eliminate all that rampant duplication in your sample code for this challenge.
 withEachRow("Select * from Users", function(rs){
	fldValue = trim(rs("fldValue") & "")
	fmtType = ucase(trim(rs("fmtType") & ""))
	if fmtType="N" and len(fldValue) > 0 then
	if not isNumeric(fldValue) then
	 appendErr "Invalid Number: " & fldValue
	end if
	end if
	if fmtType="D" and len(fldValue) > 0 then
	if not isDate(fldValue) then
	 appendErr "Invalid Date: " & fldValue
	end if
	end if
	if rs("Required") and len(fldValue)=0 then
	appendErr "Field is Required: '" & rs("fldTitle") & "'"
	end if
 });
 //hey look, used it again, wow, isn't it flexible, it even works with your code.
 withEachRow("Select bla bla", function(rs){
	sql2 = "UPDATE userFields SET fldValue='" & trim(request("fld_" & rs("itemID"))) & "' "
	sql2 = sql2 & " WHERE userID=" & userID & " AND rptItemID=" & rs("itemID")
	stdConn.execute(sql2)
 });

 //I'm betting you could also stand to use this one a lot.
 function withEachColumn(aRow, aFunction){
	for(var index = 0;index < aRow.Fields.Count -1; index++)
	 aFunction(aRow[index]);
 }
''I rarely use integer-indexed loops. To me they generally indicate a yellow alert.''

then say..

 withEachColumn(aRow, function(aCol){
	hout("<th>" & aCol.Name & "</th>");
 });
''I usually loop through the data dictionary, not map arrays. Plus, maps by definition do not define order, which generally creates problems. Looping thru maps is also a yellow alert.''

there's all the external evidence you need, if you don't see it, then you're fucking blind, because your program is rampant with duplication that these two simple HOF would eliminate entirely.

''Below I argue that some of the bloat is due to the design of Microsoft's API rather than an inherent fault of non-FP itself. One may be able to build wrapper libraries, but I didn't bother in the example.''

Moved discussion to ResultSetSizeIssues.

----
PageAnchor: code_management

In your Non-FP version, you increase complexity of a program by introduce a variable "rs" which is visible for whole method, but is only valid for uses inside the loop. In FP version the uses of "rs" is scoped to inside the block. There can be no mistake of referring to this result set again after it's used. This may be small change, but it's C/C++/Java or whatever languages with external iterator's good practice to scope the iterator to only where it's used. For example, it's good practice to code.

 for(int i = 0; i < length; i++){
	....
 }
than

 int i = 0;
 for(i = 0; i < length; i++){
	....
 }
So FP approach reduce only small code size but also reduce code complexity. How is that not significant. As I state FP is not just significant because it can only reduce code size. FP is significant because it both reduce code size and increase code comprehensibility/abstraction.

''I agree that is a nice feature, but I find it hard to call "significant". Again, you seem to be offering solutions to problems that I generally don't encounter in practice. I won't say I have never encountered problems from "leaky" loop scopes, but that probably would not make my top 10 list. -- top''

''It is probably possible to have a dynamic language where the scope is only in the given block. Thus, any variable created in a While loop will only be visible to that loop. But it may still clobber existing variables created before. But it is possible to have a language in which inner declarations don't clobber outer ones if explicitly declared.''

For the record FP version can also be made to read the whole Result set to cache first.

----
I am not against dynamic typing or agile language like ruby, I love it. Yet SQL made by concatenating string together has the same bad thing; You get the dynamic behavior that you can not reason about.

For this SQL :
 Query("SELECT * FROM employee where name = ?", name_param)	#Prepared statement
 Query("SELECT * FROM employee where name = '" + name_param + "'") #Concatenated String
In case of Prepared statement the query will work no matter what value are binded to the '?' place. And you can always be sure that it's operation is to do a query on employee based on name.
In case of concatenated string, it will work ONLY IF the name_param does not contains any ' or comment character for SQL, any statement can even be added by escaping the string and put ";" to separate a statement. You can not be sure what execution path it is for this SQL unless the name_param is know. This is not kind of Dynamicness one will ever want. Or could you give an example of where that is desirable?

Clause addable is always possible with prepared statement. PreparedStatement is not something that has to created at compile time. You can create them on the fly. For example, this prepared statement can be created at runtime:
 SELECT * FROM employ WHERE name = ? AND phone = ? address = ? AND ... # add as many "xxx = ?" programmatically.
And you can after that bind each '?' to the value. And there is no chance that the query result will change just because some of the data has '#' (comment character) in them.

This is the same objection I have for eval.
Suppose your eval statement is to do assignment to variable. So you create this:

 def eval_append(var_name, var_value)	#<var_name> is a string, <var_value> is from DB and never know.
	eval var_name + " = '" + var_value + "' + 'foo'"	#result in, for example, eval "x = 'bar' + 'foo'"
 end
Now this would work fine for

 eval_assign("x", "baz")
But it will not work this string

 eval_assign("x", "boom' ; 'x")	# assume ";" is statement separation. now we have: eval "x = 'boom' ; 'x' + 'foo'"
I can never think of the reason you will want this. you want <var_name> to always be string you can assign to something no matter what it's value is. (Or do you like a language that "'x = y' works as long as y does not equals 0, 57, and 12357"?)
You can not really be sure, with eval and string building, that the execution path of the data can be determine based on only the code you wrote.

"Code and data are the same thing" is fine for me; to the extent that I can be sure that what I want them to do to be data doesn't suddenly choose to behave itself as code without my decision.

There is a different degree of dynamicness one feels comfortable with. For me, I want dynamicness of code that is reasonable by looking at the code. I will never want some undetermined execution path to come just because some weird data enter in to the systems.

Secondly, Eval is hard to play nice with itself. In HOF, one of the best convenient is that a function can take another function as parameter and return function as a result of execution. Can Eval take another a string that is the result of another eval to use in its code, or return a string for another function to use in eval statement? Sure it can but, with the problem that code for eval is just string, you will have to double/triple escape strings depending on where you want it to really behave as string and where you want it to behave as code; which is confusing process to get right at least.

''As far as prepared statements, I don't care either way. I am tired of hearing about them. Use them if you want and if it affects the code size count in the end, I will accept an adjustment to compensate.''

PS does not reduce your code size, it is just safer to use than string building. Just like HOF and Eval.

''As far as Eval going bad, It has not been a problem for me. I focus on where actual problems occur in practice. If I needed to use Eval a lot per app, it might start to become an issue, but I don't. It works just fine for the few places that extra indirection is needed. And, I have not figured out how to store HOF references in a database. But, I can put function names in a database. (It is not an issue for #6, I would note.) But generally I end up putting expressions in the DB for most Eval uses that I recall, not function names alone.''

So now you lost the claim that putting function dispatch in DB is across language; you can't eval TCL code in Python.

''I'm not sure what you mean. I do it out of editing convenience and OnceAndOnlyOnce. If you put the same in code then you have to carry the primary key in the code also to match up with the table, which contains the attributes. It then becomes a one-to-one relationship, which is something to generally avoid. For example, to add or subtract something from the table requires you to remember to do the same to the code-based list to keep them in-sync.''

''For example, if you have a menu system where the menu descriptions and nesting are stored in a table(s), then we probably want a way to associate the menu items with behavior. If we cannot put function names or expressions in the table, then somewhere in the code the primary key probably needs to be duplicated:''

  // duplication example
  switch on menuID
	case 234: functionX(...)
	case 432: functionY(...)
	case 532: functionZ(...)
	otherwise: error(...)
  end switch
''If it is in the menu table instead, we don't have to duplicate the key and remember to keep them in sync. Thus, it is better OnceAndOnlyOnce.''

It's already stated that we can put the function name in DB and use reflection to get the function to call. Don't you remember? You said "If we cannot put function names or expressions in the table...". We can put function name in the table and use reflection to get the function to call, no duplication example above will occur. When I said about you

''Reflection is a different thing than HOF and closures. You keep adding features to solve problems. You need three different features to do what Eval does. Plus, what if we change it from putting function names in the DB to '''expressions in the DB'''? Now you need four features to do what Eval does:''

* ''HOF''
* ''Closures''
** I don't know if anyone really separate Closure from HOF. Closure happen as soon as you created a lambda (anonymous function). Language that support passing function as first class object but has no closure is supporting half of FP. Just like language that support eval but doesn't have uplevel feature (And that means you need more than one feature to fully use Eval, either).
* ''Reflection (perhaps not an FP feature)''
** The reflection I used is only looking up the name and turn it to function object to be called. HOF means having function as first class object. So I don't see why this feature would not be possible in all FP language. (If all function is object, there is no reason ability to lookup that object is not possible, right?). And for the record I can build my own hash table to store function-name-to-function-object, no need for reflection here. I only use reflection in this case because the language already do this book keeping for me.
** ''Why not just use a case statement then? Wouldn't a case statement allow more of your preferred compile-time checking than a map array?''
** You cannot dynamically add case choices. And as I stated, in practice I will use language runtime's function table so I don't keep track of available function myself. And why would I want to use case statement which look like this
 switch(f){
 ..case 'foo': foo();
 ..case 'bar': bar();
 ..case 'baz': baz();
 }
compare to
 (funcall (symbol-function f))
''Or''

  eval(funcName)
''And it scales to parameters, expressions, etc.''

* ''Eval''
** I didn't use Eval at all. Where did I use it?
** ''As stated above, if we wanted to store expressions in the DB.''
** Ahha, then sure, we must use eval. Because the requirement is "because we want to use eval". I can't argue with the requirement then. But let's consider first why one would ever want that. You need to show me '''specific use case''' first before we can discuss further.
** ''I described an example or two somewhere around here, such as the data conversion/validation utility. I'll see if I can find it. A fairly common one is a menu table where menu options call expressions, usually functions. Sometimes the functions may need parameters. I've seen other developers besides me use such.''

* And after all, I don't see why needing more than one feature to do work is bad, as long as they all work with each other smoothly. That's why we prefer orthogonality of features. HOF should not need to have ability of reflection because that's reflection's work. Eval can do anything, but half-ass of it.
** Eval can some do HOF but you have to make sure that data is never treated as code; and not to mention that you got no help from IDE now. And No compiler syntax checking. Yes I knew dynamic thing, but I never know why you don't want compiler to catch syntax error for you, if it's syntax error, it's always syntax error. It's not the same thing as Dynamic vs Static Type stuff. Even Dynamic type people will surely want to check that their syntax is correct.
** Eval can do simulate closure, at the cost of jumping through uplevel, upvar stuff. And it cannot capture the binding of variable that goes out of scope either.
** ''Why would we need that? What is an example?''
** Eval can do some of reflection, But only getting variables and methods by names; it cannot query available methods, get class of object, etc. And this reflection feature of eval is the same feature it simulate closure. So you still have the same problem.
* So I prefer many small features which work well together. Than one feature fits all you are trying to use. Top, you know how to use Eval, but you misuse it. Not that I'll never use eval in my programming life. But I know to use better, safer, more comprehensible features when possible. I only use eval when absolutely needed and remind myself of it as a smell in my code.

I didn't go out of my way at all to make an Eval problem example. It's a common mistake in Eval. You want to passed both data and code to another function; this is possible in both HOF and Eval. But in either approach, you want it to behave as what you want when you want it. If you are intending to use it as data now, it must only be data; if you want to use it as code now, it must behave as code. Eval doesn't give you that. See from above? I intended to use <name_param> as data. But just because that data contains some character, it can't be used!

''Are we talking about security or programmer convenience? Assuming database security is done right, the hacker should not be able to munge function names or expressions stored in tables anymore than they could munge the source code.''

Grrr.. I'm neither talking about security or programmer's convenience. Security hole is only one of many results of using eval this way. Using Eval, you cannot be sure that which part of the string will be data and which part will be code. The user of the system may not be hacker. But they just want to type " ' ;" as part of the data! Why can't they do that? And if talking about security. With eval/string concatenated SQL. They don't need special permission to bomb the system. They don't have to munge with the expression in DB; they just type the malicious expression directly in the User interface you provided them!

''You seem to be mixing up different issues. The Prepared Statement issue is generally unrelated to Eval. The code that Eval executes does not necessarily have to come from the user. Let's evaluate specific UseCase''''''s of Eval. -- top''

The code that eval execute doesn't have to come from user. But the data WHICH IS PART OF THE CODE that is passed to eval in someway has to come from user. Sure the following code does not come from user:

 data = ....		  // get data from somewhere
 code "print '" + data + "';"	// print the data as string (see those two 's?).
 eval code ;
But did you notice that data is actually PART OF THE CODE? If data is, Say, " '; print = 'some more". You will end up with a side effect that only happen because the data contains a single-quote which escape the string literal you intended. But looking at the code above, Can this flawed can be easily found by just looking at the code, I don't think so. Will simple test case discover this? (most people will just pass in normal data, no "'", as test data).

''I don't see how the above is significantly more likely or less testable than any other dynamic-language bug. You keep talking about how risky eval is, but again it has not been a source of big problems in my experience. I don't need it that often, and when I do need it, if one exercises a little care, it does its job. You keep insisting that brand X is going to turn my laundry green, and with more than a decade of use, my laundry hasn't turned green. I can think of only one app where the user created the expressions to be evaled, and it was a local-use app with two power-users, not something all over the company. Most of the rest were either internally generated, or came from a ControlTable.''

Ok, is there any syntax error in this code?

 eval "print('" + data +"');"
I bet you cannot tell me whether there is syntax error or not until you know what the data is, correct? Syntax error because of data? Good?

''I don't understand what you are trying to get at. I restate, in practice, Eval has not been the boogey-man you make it out to be. Quotes and stuff may tend to trip you in particular up, but that does not mean that they trip everybody else. Different people are tripped up by different things. -- top''

----
Note that the #6 example does not use Eval. Perhaps the Eval discussion should be moved.

''Move it to DynamicStringVsFunctional if you'd like.''

That is also TooBigToEdit. When the dust settles, we need to clean these discussions.

----
There seems to be a reoccurring pattern to some of our differences in opinion: I practice more SeparateIoFromCalculation and don't mix things that don't have to be mixed. For example, loops are simpler under such separation because preparing the loop and performing the loop become separated issues. I tend to loop on prepared temporary result sets, not direct active behavioral interfaces (such as files or pipes). Similarly, error handling seems simpler because high-risk activities are separated from low-risk activities such that recovering from high-risk activities is then much smaller in scope. It tends to be part of my data-oriented-interface view. Communication between "parts" of algorithm is via data structures (preferably tables in many cases) instead of via behavior and behavioral interfaces. Your examples and demonstrations over-use behavioral techniques in my opinion. You try to do too much at once. You nest processing that does not need to be nested (at least not in my domain). It is not that FP does not help in general; it is that it does not help much when doing things my way. Now, whether my way is "bad" is another topic. But it's similar to some of the reasons my views conflict with OOP also, which tends to be behavioralistic.

PageAnchor: coding_thumb_rules

Here is a summary of practices that would reduce the frequency of need for many of the things you FP's try to sell:

* SeparateIoFromCalculation

* Do things sequentially unless there is a compelling need to intermix them live

* Don't open too many files/sockets/connections at the same time unless there is a real need. Buffer output to a database, table, or local file(s) so that bunches of things bound for different destinations can be written out sequentially if need be. This greatly simplifies the code, restart management, and debugging.

* Use tables for non-trivial structures, not nested structures.

* Use result sets instead of cursors. The need for cursors should be relatively rare, and put on the database server when they do occur if possible.

Now, I will grant there are occasional times where one cannot follow these principles and FP may indeed simplify those particular spots. However, I question whether it is wise to double our tool set just to improve a very small percent of the total code. One should not violate KISS unless there is a compelling need to gum up the works.

-- top

''"Now, whether my way is "bad" is another topic."''

No, I think it is part of the same topic. If I'm understanding it right, it sounds incredibly inefficient for many applications. What do you do when you encounter a performance problem? Do you then deviate from the techniques that you advocate, and if so, what techniques do you use then? -- DanMuller

''Machines are cheaper than humans (at least in the US). And, in my domain the bottleneck is usually the network, not the CPU. Also, the RDBMS can do a lot of auto-optimization for you if you let it. Given these conditions, I will happily let the machine do the grunt work so that I don't have to. If I encounter a specific performance problem, then generally a little rework is all that is needed. Most performance problems I witness are due to bad schemas and/or bad queries, and not from principles such as SeparateIoFromCalculation. -- top''

I'll willingly agree with all those points, and work from similar assumptions myself. But the key words are "usually", "can", "generally", and "most". There are exceptions that I find to be infrequent but far from non-existent, such as large batch processes that require a sophistication of processing which the query engine can't handle for me. So what do you do in those cases? Just tell the user that they have to live with the long processing times incurred by all the data copying? Or is this when you resort to cursors? (Note that I'm not trying to enter this debate in any way, I'm just curious.) -- DanM

''I never said never. {Oops, I caught myself in a recursive lie here.} If you really need client-side cursors, use them. The FP wrapping techniques suggested seem to only materially matter if there are a '''lot''' of cursor loops. It appears that one is sufficient for that case. That one loop may indeed be 3 more lines than an FP version. But using multiple paradigms to tends to bring diminishing returns and increases the expense of staff. It might be great if you are the receiver of the paycheck, but businesses do weigh those diminishing returns which are cancelled out by staffing issues. The numbers are against it.''

 Benefits of each additional paradigm (illustrative only, not based on actual studies)

 1 | ***********
 2 | ***************
 3 | *****************
 4 | ******************
 5 | *******************

 Costs (hiring costs, staffing costs, staff transition delays,
	training, debugger cost, etc.)

 1 | ***********
 2 | **************
 3 | *****************
 4 | ********************
 5 | ***********************
Businesses want PlugCompatibleInterchangeableEngineers. They will only abandon this goal if you can show a compelling benefit. Going from 3100 lines of code to 3097 lines of code is not even near "compelling". Related: HackerLanguage. -- top

Well, much of the discussion on wiki is about how to improve software engineering, not how to kowtow to bad practices propagated by inertia in management and vendor products. Anyway, thanks for the answer, but no thanks to all the extra stuff (including invented graphs) that came with it. -- DanM

Business managers sometimes piss me off also, but your assumption that most business practices are inherently wrong is misguided in my opinion and in this FP case I agree with their general rejection of FP. A good case for it has not been made in the domain. Its features seem better suited for systems software and communications tools. I will agree with this though: if staffing/training/education and language-complexity are not factors being weighed, then I could possibly side with you. I am just trying to put myself in the shoes of those who decide on languages and methodologies from a capitalism perspective. Whether that is the "right" perspective is perhaps another entire philosophical topic in itself. But, hopefully we have identified and narrowed down the areas where we differ. -- top

----
Just another invented Chart

 Benefits of each additional paradigm (illustrative only, not based on actual studies)

 1 | *******
 2 | **********
 3 | *************
 4 | ****************
 5 | ************************

 Costs (hiring costs, staffing costs, staff transition delays, 
	training, debugger cost, etc.)

 1 | ************
 2 | **************
 3 | *****************
 4 | ****************
 5 | ***************
Now we all should be using Multiparadigm language shouldn't we?

NOTE:
* First of all the graph is to show that no one, including me and Top, could just show some made-up chart (Which he said wasn't based on any study) and then derive any conclusion from it.

* ''They were meant to illustrate anecdotal evidence. Just because they are visual instead of verbal does not necessarily imply a plot to mislead. Are you an anti-visual bigot or something? Verbal evidence can be anecdotal, but visual cannot? How recon? Braggarts of OO and FP have failed to show significant improvements in the domain of custom biz apps. It is a lot of hopla that adds maybe a 3% improvement at best at the expense of say a 30% more complicated tool-box. Why complicate a toolbox by 30% for a 3% gain? Not a good deal if you ask me until you somehow like to carry around and hire experts in bloated toolboxes.''
** It is not that I believe visual less than verbal. It is that I don't believe both. Just because the ANECDOTAL visual or verbal say multiparadigm add 30 complexity for 3% gain and I should believe it? Why don't you believe my anecdotal visual then?

* Secondly, I somehow believe that this Chart might also be true. Given that each paradigm works together consistently.

* ''Please clarify.''
** See below

* Training cost doesn't depends on how many paradigms a language is oriented to, it depends on how well those thing, even in same paradigm, integrate together.

* ''Please clarify.''
** Lisp have both FOP and OOP, yet you don't have to learn a new syntax to call object's method in Lisp. You can also use all the FOP's usually operator (map, apply) on Object.
** ''I am mostly referring to using a paradigm effectively, not so much syntax.''
** Being able to utilize the good part OOP and FOP ''At the same time'' is not effective? What does it mean to use something effectively then?

* Debugger cost depends on Environment; Multiparadigm Lisp built-in debugger that can inspect and change code on the fly on unexpected case is cheaper and better than single paradigm C, which just crash.

* ''I am not a C fan either. But this is not the topic in which to bash C.''
** The point is that there is more to justify a language's difficulty than the number of paradigm it self.

* You seems to think that multiparadigm language must look like a patchy language where new syntax is kept adding to the core simple syntax. Lisp, Ruby is quite multiparadigm, yet each feature integrates in nicely.
** ''Again, syntax is only part of it.''
** What's the other part? Please clarify.
** ''It is mostly about knowing when, why, and how to use techniques effectively.''

* ''You are talking languages, not necessarily paradigms.''
** I am talking that the complexity of the program does not come from how many paradigm it has, but from the lack of ability to represent each programming paradigm in the way they are supposed to be expressed.
** ''I think you are selling Lisp, not multi-paradigms.''
** I'm not saying Lisp specific. Smalltalk and Ruby also enable DSL to be as expressive as possible.

* It's not that the training cost increase wrt paradigm of languages; Have you ever been in training course where the agenda list training in order of programming paradigm?

* ''Please clarify.''

----
Also note there is the cost of those who make big messes for the sake of job security. The more paradigms you can mix up, the more weapons you have for job-security. I have seen cases where boredom led to MentalMasturbation, and multiple paradigms just gives a dude more dicks.

''You seem to have a distrustful view of your employee, don't you think?''

In the current work-world, there is not a lot of incentive to write readable code. I know one guy who claims one shouldn't comment code because once he saw his comments confuse the optimizer hint system, and the boss buys that excuse. -- top

''It is also possible that someone will use DBMS just to kill boredom. It is also possible that someone will use Eval just for mental masturbation. It is also possible that someone will program in TOP just for mental masturbation.''

Some indeed do. But it is a matter of quantity. Code where 2 paradigms are abused is better than code where 5 are abused.

''[Ehh... No, if it is a mess, it is a mess, even if it's a single paradigm program. Quantity wise, 10 line of code will be better than 20 lines of code. And 2 lines of code will be better than 10 lines of code. Yet we know what PerlGolf is And we know how Verbose XML is. And if many choices is bad, then you shouldn't only count paradigm, but count library out there. How many thousands of free libraries are there for Java/C#/Perl? Any programmers can download a library and abuse it in his program. Why should we allow that? Abusing uses of Library is no better than abusing uses of paradigm. Why don't we have a language where one cannot create a library? What's the difference? Disregarding mental masturbation, who force him to use a library where he doesn't need? Who force him to use a paradigm where he doesn't need?]''

'' And it is certain that, in the current work-world, there is not a lot of incentive to know enough to program TOP effectively (Only topmind seem so). I once saw a guy trying to simulate multi-dispatch using Database Table. Does that also mean we should discourage TOP too?''''

What is wrong with using tables for multi-dispatch? I am not saying it is always the way to go, but I would like to know specifically why it didn't allegedly work. I've worked in shops where tables where heavily used for such and they were productive and reliable.

''Because the language already provide the function?''

No, you have to build the dispatch matrix from scratch. And, tables are better OnceAndOnlyOnce because an OO version repeats the "cell header" over and over again (method declaration).

''Because you are doing things unconventional to your codeveloper?''

So the status-quo is a self-fullfilling prophecy? ExBase shops often used expressions in tables. It was not "unconventional" to heavy ExBase people.

''Because your dispatch is based on unsafe eval?''

Not this anti-scripting debate again. Besides, one can use case statements instead if you really want static "comfort".

''Also, there are shops that use multiparadigm and works just like your shop that table works. So what decides that multiparadigm should be discourage while TOP should not?''

Tables and procedural complement each other well. Thus, you have only 2 paradigms which generally don't fight over territory.

''Higher order functions are used to do something like this''
	SchemeLanguage Code 	
	(map-query "select * from foo where bar = baz"
			(lambda (x) (printf "~a~%" (* x x))))
And this does what? what is "~a~%"? The SQL does not change, so why run it in a loop?

''It's not in a loop silly the first argument is the query and the second is the body of the loop. its lisp format strings''

What use is it over more conventional techniques? Stuff like "~a~%" is not very self-documenting, so I'd knock points off for that.

(Who has been doing screwy grammar editing of late? Please be a bit more careful.)
----
How does adding one keyword Lambda, and removing restrictions on function passing have on complexity vs having to add a keyword for every control flow abstraction.
Lets say i have a table and i want to map over the tuples therein
imperitive code
	 for(i = query("select (quux.foo,spam.bar,eggs.baz) from quux,spam,eggs where quux.foo = ? and spam.bar = eggs.baz","abc");
		!done(i);
		next(i))
	 {
		printf("%i,%i,%i",i.get[0]+1,i.get[1]+22,i.get[2]*3);
	 } 
Functional code
	 foreach(query("select (quux.foo,spam.bar,eggs.baz) from quux,spam,eggs where quux.foo = ? and spam.bar = eggs.baz","abc")) (foo,bar,baz) => {
		printf("%i,%i,%i",foo+1,bar+22,baz*3)
	 }

''What is being demonstrated here?''
 
----
For what it's worth, I was upset by the on-all-sides-argumentative and fruitless conversation on this page, and decided to do something about it. http://github.com/ods94065/challenge-six is my tilt at the windmill. The solution uses a grab-bag of Lisp tricks, including some FP, to shave off somewhere between 9-30% of source code size depending on how you measure it. Not bad for a simple demo app IMO. -- Owen

''Looks good to me. Just a minor nitpick though. C++ has made some changes recently (i.e. C++11) because they've "seen the light", as it were, on the value of expressions. You can now do the following.''

	std::vector<std::string> x()
	{
	    return {"foo", "bar", "baz"};
	}

Continued at ChallengeSixLispVersionDiscussion...
---- 
CategoryExample, CategoryBusinessDomain