Continuation of portions of ComplexityOfOutputtingDuplicateTuplesInTutorialDee:
----------
Your point in the side-effects and XML examples is not clear: Many query languages ''do'' have side-effects, such as SQL's UPDATE statement. 

[SQL is not a single language. SQL is made up of several smaller languages, covering various aspects from transactions and access control to data definition and data manipulation. Only the stuff you are allowed to place inside the 'SELECT' statement is truly part of SQL's 'query' language. While I do not practice SQL very often, I do not recall the ability to embed UPATES into the SELECT statements, even though vice versa is okay. If my recollection is correct, then SQL is properly a 'layered language' with respect to SideEffect''''''s. We are suggesting that RelationalModel query languages should, similarly, be layered with respect to presentation details (such as ordering, or XML format)... i.e. such that these features cannot complicate the 'query language' proper.] 

* Fair enough. Can you find a way using layers to make it so that one can omit primary key columns in Rel's results withOUT duplicating the SELECT list and without having to write imperative PRINT loops? More on layering below.
* [The EXPORT example, from way back in BagAtational, already showed you how to avoid imperative loops and ''most'' column-name duplication. Intermediate naming of queries might allow one to avoid further duplication (but I'm the wrong guy to ask about Rel).]
* Drawbacks of EXPORT discussed below.

For example, SQL could add syntax like: SELECT * TO FILE 'data.txt' FORMAT XML FROM foo. ExBase was sort of like this. 

[SELECT statements can be embedded within one another. So one question is: what, exactly, would that mean (in terms of semantics, performance, concurrency issues) in the event it was embedded within another SELECT statement? Similarly, SELECT statements may be used to define UPATEs and VIEWs and such. What would SELECT TO FILE' mean in those cases? Additionally, SELECT statements are also performed over the wire, between unlinked FileSystem''''''s What would SELECT TO FILE mean for purpose of security and accessibility issues?] 

Like a prior suggestion, only the "outer" final statement should perhaps be allowed to create output. I would note that it's not conceptually much different than "SELECT ... INTO myNewTable..." of some existing dialects of SQL. It would be interesting to test an inner "into". As far as security, file writing should probably be disabled by default for DB users. I see it more useful on a DesktopDatabase and DBA's security level.

In SmeQl, I'd probably have a SAVE(...) and FILE(...)  or Export(...) operations to create new tables from queries and files, respectively (see TqlDataAlterationOperators). They'd only "work" as the outer or final query, though. The final/outer operation has more leeway. This is closer to the "layered" approach you talk about above.

Most non-proprietary info processing tools can import and export standard formats. That's how they talk to the outside world. 

[Sure. So can RelProject. It simply isn't part of the 'query' language. Export doesn't need to happen in the middle of a RelationalModel query. Sometimes import is needed (i.e. lazy import, or to handle updates) but creating 'new' links shouldn't be part of the query itself, and so linking of external data resources is also expressed outside the query language.] 

* I didn't say it had to happen in the middle. Just something more convenient, less imperative, and less duplicative than your loop proposal.
* ''How many '''more''' times do I have to point out that the loop was an illustration?  It's not a requirement.''
* How many more times do I have to ask for its replacement? If you agree it's poor, then show the "right" way.
* ''Myself and others have '''repeatedly''' mentioned EXPORT as one example.''
* I don't know where the debate text went, but my main complaint about EXPORT is that you have one command for SELECT (or its equiv.) and a different one for EXPORT and they do ''almost'' the same thing. It would be better feature factoring to make them one of the same to avoid confusion. "Almost the same" commands/features are a design smell that should be avoided. -t
* ''SELECT (i.e., relational projection) may be used as the input to other relational operators.  EXPORT may not.  That, in itself, is sufficient justification to conceptually -- if not syntactically -- separate the two.''
* That's a description of the nature of the proposed feature, not of the reason behind it. "Conditioning" a table/result-set for use in another system/sub-system and "formatting" overlap heavily in my experience. I see no reason to force an arbitrary line between them. In other words, the idioms for "adjusting" for internal systems or uses and for "output" are not different enough to justify two different commands in a query language.
* ''In other words, you have no logical reason to combine SELECT and EXPORT, whilst I have a logical reason to separate them.  Note that in separating them, there's nothing that precludes designing a language in which they can appear together in a single clause.  Using SELECT as the input to EXPORT may be as simple as preceding the latter with the former.''
* I'm not following what you claim is "logical". SELECT has feature set A,B,C,D,E,F,H while EXPORT has feature set A,B,C,D,E,F,G. "H" is the ability feed to another table, and "G" is the ability to have a "bag" result, and the rest of the letters being typical column preparation and calculation idioms. Based on these two differences, you create two different commands. As interface design, this is generally to be avoided. It's almost like having a Print() statement and a Print_Black_And_White() statement. It's AttributesInNameSmell. And your doubling-up "both" option is confusing to users. I'd generally agree to having an "allowBag" modifier, or at least complain far less about it.
* For formatting reasons, I shall answer in the block below:
----
''I'm not following what you claim is "logical". SELECT has feature set A,B,C,D,E,F,H while EXPORT has feature set A,B,C,D,E,F,G. "H" is the ability feed to another table, and "G" is the ability to have a "bag" result, and the rest of the letters being typical column preparation and calculation idioms. Based on these two differences, you create two different commands. As interface design, this is generally to be avoided. It's almost like having a Print() statement and a Print_Black_And_White() statement. It's AttributesInNameSmell. And your doubling-up "both" option is confusing to users. I'd generally agree to having an "allowBag" modifier, or at least complain far less about it.''

In TutorialDee/RelProject, the RelationalAlgebra operators are fully exposed, such that there is no SELECT operator per se -- at least not in the SQL sense.  For example, given RelVar''''''s a, b, p, q, and s, the following are all valid relational expressions or queries:
 p
 a MINUS b
 TCLOSE q
 SUMMARIZE s BY {x} ADD (SUM(balance) AS total, AVG(balance) AS mean, COUNT() AS n)
 p WHERE x = 3
 a INTERSECT b WHERE x = 3
 (a {x} UNION b {x}) MINUS c
 a UNION b WHERE x = 3 {x, y, z}
Note that only the last two expressions feature an explicit projection (specification of columns, duplicates eliminated in the result).  Many times, we don't need to perform a projection.  Sometimes, we may have reason to project columns in the interior of an expression, as an intermediate result.  The second-last expression, above, is an example.

This is quite different from SQL, where a SELECT always requires a column specification, either explicitly or by wildcard, immediately following the 'SELECT' keyword.

In most cases, the relation that results from evaluating an expression/query will be used directly by a client-side application.  In that case, no EXPORT is needed.  We send a query to the DBMS and do whatever we like with the ResultSet that comes back.  In other cases, we might wish to assign the relation to some variable for later use on the server-side.  E.g.:
 p := a UNION b WHERE x = 3;
An EXPORT is only needed if we wish to export the relation (or ARRAY) to some '''external target.'''  For that, there is an EXPORT statement.  It only provides a means to (optionally) exclude columns; there is no need to specify their inclusion, because that can be done by an explicit relational projection as shown above.  EXPORT also specifies the target (file, device, ODBC/JDBC table, etc.), which obviously has no meaning within the relational algebra.  

For example:
 EXPORT r FORMAT CSV TO "myfile.csv";
In the above, 'r' can be an arbitrary relational expression.  The following are all valid:
 EXPORT a UNION b FORMAT CSV TO "myfile.csv";
 EXPORT p WHERE x = 3 FORMAT CSV TO "myfile.csv";
 EXPORT a UNION b WHERE x = 3 FORMAT CSV TO "myfile.csv";
 EXPORT a UNION b WHERE x = 3 {x, y, z} FORMAT CSV TO "myfile.csv";
Note that only that last example has an explicit specification of columns, i.e., {x, y, z}.  This is part of the relational expression, not the EXPORT statement.

We might wish to exclude a column from the output, in order to produce a bag with explicit duplicates.  That is part of the EXPORT statement, because it has no meaning (or value) within relational expressions.  We can do things like this:
 EXPORT a UNION b EXCLUDE q FORMAT CSV TO "myfile.csv";
 EXPORT a UNION b WHERE x = 3 {x, y, z} EXCLUDE z FORMAT CSV TO "myfile.csv";
Now let's look at features:
* Evaluating a relational expression produces an arbitrary relation.  The expression may or '''may not''' involve explicit specification of columns.  If an expression performs projection, it may occur (possibly multiple times) in the middle of an expression (e.g., 'a {x, y} UNION b {x, y} WHERE x = 3').  '''Relational expressions are feature 'A'.'''
* EXPORT sends a relation to a target.  '''Sending an arbitrary relation to an external target is feature 'B'.'''  '''The optional ability to exclude a column and thereby emit duplicates is feature 'C'.'''
So we have relational expressions with feature 'A' but not 'B' or 'C', because 'B' and 'C' are meaningless in the relational algebra.  

Conversely, and for the same reason, EXPORT implements features 'B' and 'C' but not feature 'A'.''

Note that from a user interface point of view, it's all one "query", e.g.:
 EXPORT myRelvar {k, a, b} ORDER(ASC a) EXCLUDE k FORMAT CSV TO "myfile.csv";
The fact that this is a combination of two distinct operator categories -- one being relational expressions, the other being EXPORT -- is largely invisible to the user, who need only think in terms of queries.  Conceptually, however, relational expressions and EXPORT are '''completely''' different beasts with no overlap.  Their functionality is mutually exclusive.

Therefore, it's not "almost like having a Print() statement and a Print_Black_And_White() statement."  It's more like having a Print() statement and expressions, and the Print() statement can present -- with special formatting (i.e., specify a target and optionally exclude columns) -- the result of evaluating an expression.

''First, I object to excluding EXCLUDE from queries.''

There's a distant relative available for projection (and other column selection) lists called ALL BUT.  For example, given the following RelVar definition...
 VAR myvar REAL RELATION {x INTEGER, y CHAR, z RATIONAL} KEY {x};
...You can project on x, y by inclusion with:
 myvar {x, y}
...Or by exclusion with:
 myvar {ALL BUT z}
The above examples are equivalent.  However, ALL BUT in a projection and EXCLUDE in an EXPORT are very different things.  The EXCLUDE clause in EXPORT allows you to drop one or more columns and possibly emit duplicate records.  Projection always produces a relation, which (by definition) means it will never emit duplicate tuples.

''Second, most existing RDBMS have various kind of export commands, but they run on the server side, not the client side and thus are usually of little use to typical ad-hoc query users.  That the query language has knowledge of or access to the local file system may be assuming too much. The query engine typical does not know what the end-user is going to do to the result set. In many client-side query tools one can browse the result set in a data grid and then later decide to save that info to disk, and this info is not sent back to the DB engine and it would be too late even if it did. Is Rel growing into a network OS also? Something doesn't make sense. -t''

Rel is a desktop DBMS, a server DBMS, and a general-purpose programming language with integrated database capabilities. I'm working on facilities to seamlessly link server Rel DBMSes to your desktop Rel.  That will allow you to run EXPORT locally to export data from your local database or remote databases.

''I applaud your effort, but it may be safer to walk before you fly by releasing a basic RDBMS first and get feedback on that rather than try to build the whole kitten caboodle first. And just because an integrated client is available doesn't mean it will be readily available in a production environment. But we'll see.''

* I've been getting feedback on "a basic RDBMS" since the first release of the RelProject in 2004.  Your point regarding integrated clients is true, but equally applicable to any DBMS<-->client environment.  A custom client can, of course, do anything it likes with the ResultSet''''''s it obtains from the DBMS.  By the way, it's "kit and caboodle", not "kitten caboodle".
* ''But Donkey Hodie and Elron Hubbard told me it was "kitten" at our windmill dinner. As far as a client engine, one should consider the likely possibility that it won't be available or in use at a given shop, just like existing Oracle and Microsoft-SQL-Server clients are often not used. Thus, the primary query processor won't have any control over or knowledge about what the user does with the result set, and thus EXPORT will be underutilized. That's my tool design opinion.''
* "[T]he primary query processor won't have any control over or knowledge about what the user does with the result set, and thus EXPORT will be underutilized."  Yep.  That's equally true of (e.g.) MS SQL Server's BULK INSERT command or the 'bcp' utility, for essentially the same reason.  So?
* ''Thus, in practice one ends up using the abilities (or lack of) of the SELECT clause, or it's equivalent, alone (plus local tools). Your suggestion that EXPORT would remedy some of the issues that you call "formatting issues", such as hiding employee numbers for privacy reasons in an ad-hoc report, is thus rendered moot. It wouldn't typically be available to a query programmer.''
* What "SELECT clause"?  A non-Rel, custom client of a server-side Rel DBMS can do anything it likes with a ResultSet.  It can format how it likes, output it how it likes, generate or remove duplicates, export to things we haven't thought of yet, deploy a robot army against my enemies, anything.  It doesn't need EXPORT, because its power is potentially infinite, software-wise, limited only by whatever programming language(s) the developer chooses to use.
* ''It needs a custom one instead of use existing query browsers? That's another ding against it.''
* What's a query browser?  I assume you don't mean Query Browser, which is specific to MySql.  Do you mean SQL development/administration clients like SQL*Plus and SQL Developer?  Those are specific to Oracle Database.  TOAD is mainly Oracle, PGAdminIII is only PostgreSQL, and so on.  I'd guess there are generic ones that support any ODBC (or JDBC?) provider, but I've not used them.  I assume those would work fine with Rel (once the ODBC/JDBC drivers are written) as long as they don't require SQL.  Remember that Rel implements TutorialDee, not SqlLanguage.  Also remember that DBMSes are mainly used to underpin database-driven applications, such as PHP scripts that implement Web sites or Web-based applications, C# or Java desktop/Web/WebServices applications, and so on, so the choice of "query browser" is of relatively little importance.  Anyway, I'd hope that the notion of using a Rel desktop DBMS as a development/administration client for Rel server DBMSes would catch on, in which case EXPORT '''is''' available.
* ''See ListOfQueryBrowsers. Which ones can work with non-SQL query languages, I can't say. It depends on part how much up-front parsing they rely on. As far as being used within an app language, the same issue still applies. EXPORT is not available to them anymore than the DUMP TO TEXT or equivalent in existing RDBMS are not available to most apps. Thus, '''EXPORT still does not remedy the original problem''' of being able to include and exclude columns as needed per usage point in typical production environments.''
* As was already pointed out, the "typical" production environment is a PHP/Perl/Python/C/C++/C#/Java/COBOL/CrystalReports/Q''''''likView/Cyberquery/etc. program.  Therefore, including or excluding a column is as trivial as referencing it or not.  Hiding columns in a tabular display is a feature of MicrosoftAccess; I've no doubt it already is (or can be) incorporated into other "query browsers".
* ''I'll give you the point that it's easier to ignore unwanted columns via the app language. However, there are two downsides of doing that. First is the confusion of listing a column that is not actually used. It's almost like declaring a variable but never using it. Second is the extra network traffic used to deliver the unused column. Network bandwidth in large organizations is often limited. These are not huge factors, but something to weigh nevertheless.''
** PageAnchor: formatting_01
** The client/server protocol can trivially support pulling only the columns (attributes) requested by the client, on a row-by-row (tuple-by-tuple) basis.  I don't know what you mean by "listing a column that is not actually used."  See the examples above.  There's very little "listing" of columns at all.
** ''Without detailed investigation, we don't know if it's trivial with existing standards, such as ODBC/JDBC. How is the DB server to know what to include or exclude? Your examples use EXPORT, which is not applicable to most circumstances as already described. ''
** EXPORT, like the "FOR" example before it, was intended to be illustrative of SeparationOfConcerns between relational expressions and output formatting.  It was not intended to be a comprehensive survey of all possible targets for the result of evaluating a relational expression.  The JDBC API certainly makes it trivial to design a JDBC driver that retrieves individual attributes on demand.  The contents of a row (tuple) may be cached on the server side, with individual attributes sent over the wire to the client upon request.  I haven't coded anything for the ODBC API in a while, but my recollection is that it's the same.
** ''Even if it was technically possible, it may not be activated and/or implemented in existing query browsers/tools because it's not really needed with SQL. If unchecking "show" for a given column removes the given SELECT column from the generated SELECT column list, then there is no reason to build in a second mechanism that does the same thing (column suppression) a different way. Companies don't program the same feature twice unless there is a known and immediate need. As far as SeparationOfConcerns, it's not really separate. "Adjusting" the output/transfer-data format/layout for different humans and/or computers overlaps heavily in practice. There is no reason to force a hard line between something that usually has no hard line in the field. Adjusting is adjusting regardless of whether it's for humans or machines and the type of adjustment done for humans and adjustments done for machines overlaps heavily until the point that fonts or colors come into play. If you want to go over transform actions and the probability of being done for humans versus machines, we can if you feel it will clarify this issue. -t''
** Rel requires its own ODBC/JDBC driver (all DBMSes require their own ODBC/JDBC drivers), so providing data transfer optimisation is merely a matter of building it into the ODBC/JDBC driver.  Query browsers and other clients need not be a aware of it; it's built into the ODBC/JDBC driver itself.  As for SeparationOfConcerns, projection is as distinct from output presentation as the "+" operator is from a "W''''''riteLine()" operator, and are distinct for the same reasons.
** ''Again, a given query browser may have no other existing mechanism for "excluding" columns other than removing them from the sent query text. Even if your ODBC driver had that capability, such a browser couldn't tap into that functionality without a software change. If there is not a mechanism in place in the browser to "tell" your driver to exclude a given column, then it won't happen. It's an ejection seat without an "Eject" button.''
** Indeed, some third-party query browser could be too simplistic to control ResultSet attribute displays, such that every entire tuple is transferred down the wire every time a relational expression is evaluated.  I consider such a tool to be broken -- much as various database tools are already broken in various ways and so I avoid using them. I'd hardly consider the lack of quality of ''some'' third-party tools to be a reason to deprecate the effectiveness of my tools.
** ''You misinterpreted something. The inclusion/exclusion is often controlled by the existence or absence of a given column from the SELECT statement field list. Often such lists can be auto-generated as an option. That's not the same as "not having". I thought that was pretty clear. I'm not sure where the confusion arose from and am growing frustrated here.''
** I've no idea where the confusion lies.  We seem to be talking across each other here.

** ''As far as your "+" versus "print" SeparationOfConcerns analogy, there are times where one wants to "print" to a string variable. The concept of PRINT could be reworked to be something like FORMAT( "%d,%s,%s", A, B, C, TARGET=STDOUT). If one omits the TARGET clause, then it just returns the formatted string (if there is an accepting variable). It could even save it to a variable and send it to an output destination. PRINT essentially '''combines''' output with formatting due to common '''convenience''' and historical habit. It would be easy to linguistically separate them into FORMAT(...) and OUTPUT(...) and use OUTPUT(FORMAT(...)...) instead of or as an alternative to PRINT(). It's a kind of composition pattern. Destination control and data re-arranging are two different things. If you want SeparationOfConcerns, then separate those aspects. You are splitting wrong.''
** Sorry, I must not have been clear.  "+" was intended to be representative of RelationalAlgebra operations, i.e., computations that generate a value.  "W''''''riteLine()" represents output formatting which displays or emits a value, which could include PRINT, FORMAT, OUTPUT, whatever, to an arbitrary destination.  It is generally considered good SeparationOfConcerns to separate calculation (e.g., "+", relational projection in ''Rel'') from presentation (e.g., "W''''''riteLine()", EXPORT, PRINT, FORMAT, OUTPUT, 'SELECT' column lists in SQL.)
** ''The concepts overlap. That's life.''
** The concepts are disjoint.  Their concrete realisation exhibits a superficial commonality, similar to the superficial similarity that exists between UML class diagrams and classic flowcharts -- they both use boxes and lines.
** ''No, they are not. You are mentally forcing your view of the world into concepts that don't necessarily care about what's in your particular world classification system. "Formatting" happens to have the same or similar CollectionOrientedVerbs that relational has. Adding, changing, deleting, filtering, etc. is common across MANY transformation concepts. Relational has '''no claim of monopoly''' on adding and subtracting items from a list/tuple/map, etc.''
** Formatting has CollectionOrientedVerbs?  Furthermore, formatting has the same or similar CollectionOrientedVerbs to the RelationalModel? Sorry, you've lost me here.  None of the classic writings on the RelationalModel (Codd, Date, Date & Darwen, etc.) even ''mention'' formatting.
** ''There seems to be a huge mis-communication here. I'll have to review this to plan a different way to say it.''
** Yes.  I feel, increasingly, that it's not so much that we misunderstand each other, but that we have a fundamental difference in priorities.  Indeed, I suspect that whilst most arguments might be attributable to LaynesLaw, the remainder are due to differences in priority.  One's priorities, it seems, come from life experience and are unlikely to shift under the featherweight of mere debate, even if (or especially if) the debate becomes heated or descends into insults.

* ''As far as the query browser removing columns, many control it by controlling what columns are included/excluded in the SELECT statement. It doesn't make much sense to do it on the output side only because it may result in a far wider result set coming over the network than needed. (I don't know how MS-Access specifically does it for non-MS DB's. Columns mentioned in its query builder but have display being switched off (no check-mark) seem to only create WHERE clause items, not SELECT items that are somehow later ignored. That would otherwise be confusing as hell for anybody looking at the generated SQL. I can't see any rational reason for doing it two places other than a weird language like REL that insists on output key purity.)''
* I can't see any rational reason for generating a bag out of a perfectly reasonable projection, like that weird SQL language.  What were they thinking?
* ''Why not just have a KEY_CHECK_OFF option keyword in REL to make it compatible with SQL conventions when needed? Work with the world, don't fight it. -t''
* REL is a company that makes speakers.  The name is ''Rel'' (italics are part of the name), though Rel is acceptable.  ''Rel'' deliberately defies SQL conventions where they are deemed flawed.
* ''Bags are not flawed; they mirror the real/domain world. Your idealism is what's flawed. Anyhow, we are going in circles. LetTheReaderDecide. [Removed insult]''
* Why the AdHominem attack? You can't claim you were inspired to insult by the emotional heat of the debate, because this has (until now) been a pleasant and polite technical discussion.  Anyway, given a DatabaseIsRepresenterOfFacts and neither a domain simulator nor a warehouse, it does not make sense to maintain bags of duplicate facts.  If there are 'n' identical cans of cat food on the store shelf, I will record in an inventory table/RelVar the fact that "I have 10 cans of Tasty Tuna(tm)." (E.g., TUPLE {ProductID 'Tasty Tuna', Qty 10}) I will not record "I have a can of Tasty Tuna(tm)", "I have a can of Tasty Tuna(tm)", "I have a can of Tasty Tuna(tm)", "I have a can of Tasty Tuna(tm)", "I have a can of Tasty Tuna(tm)", "I have a can of Tasty Tuna(tm)", "I have a can of Tasty Tuna(tm)", "I have a can of Tasty Tuna(tm)", "I have a can of Tasty Tuna(tm)", "I have a can of Tasty Tuna(tm)".  In logical terms, using duplicates in a bag to represent a quantity isn't even accurate, because a fact recorded 10 times is logically equivalent to a fact recorded once.  So, ten records of the fact that I have a can of Tasty Tuna(tm) means, in logical terms, only that I have a can of Tasty Tuna(tm).
* ''You are right, my AdHominem attack was unwarranted here. I was still steaming from another topic and accidentally cross-mixed my anger. I have removed it. My apologies. As far as duplicates, in that situation as stated I agree with you 100%. However, the nitty-gritty of reality sometimes mucks things up. I gave some examples in BagNeedScenarios and I stand behind them still. A common theme there was that your side keeps assuming that they have the authority and time to re-engineer everything in big companies to clean up bad practices and fix broken devices. -t ''
* I believe "my side" has made it clear that your BagNeedScenarios are trivially addressable (or, as in the case of #4, too contrived to be generally-applicable) by a true relational DBMS, purely at the DBMS level itself.  No engineering is required outside of using a true relational DBMS.  (Of course, if you don't intend to use a true relational DBMS at all, then this entire debate is moot.)
* ''What exactly do you mean by "contrived"? Rare? Having to squeeze a lot of info into a small area is a fairly common need. True, it's less likely these days as we have more capacity and options due to the march of technology, but it's still an issue. Suppose you had to buy 10,000 flash drives to mail to potential clients in a marketing campaign to marketers with actual data for a given city as a sample. Obviously you'd want cheap flash drives to save money on that quantity, and reducing capacity to the bare minimum required is one way to achieve this. I agree that #4 is probably less common than the rest, but still a realistic scenario. Please clarify "no engineering". No effort and no delay? Not. I dismiss a specialized DB engine for all situations for reasons already given. That can be a lot of effort and re-training complexity to go through JUST to make the purity gods happy. The simplest thing is just to drop the primary key in the existing DB engine when making the stick.  [EditHint: move and link this back to scenario topic when cooked.]''
* "Contrived" means the scenario is constructed purely to make a point, but the point is not inherently valid.  If lack of disk space (and CPU horsepower) force us to accept duplicate rows, then even less disk space and CPU horsepower will force us to (e.g.) use pen and paper. Such "imagine <x>, therefore <y>" scenarios are infinitely constructable.  It's only a few doors down from "imagine that we have to include duplicates in tables or the boss will shoot us with a gun, therefore support for duplicates is needed."
* ''I'm not sure I'm following your argument. As constraints narrow, our options usually narrow. There may indeed be a DiscontinuitySpike point if things get too squished, but that doesn't mean there is always such a spike. Dumping the primary index gives a trade-off scenario to consider. I added a dialog scenario in BagNeedScenarios under PageAnchor "scenario_four_alpha". Hopefully this will clarify stuff. Again, running into constraint ceilings is fairly common in my experience. Companies usually just don't fling upon their wallets to buy new equipment or resources until there is a clear-cut problem that multiple managers agree on (except maybe military contractors :-).''

{RelProject works, today, which is more than you can say for any project you've ever advocated. Why don't you start ''crawling'' and build your SmeQl.}

''I'm working on a different open-source database tool project right now, kind of a web-based half-Toad/half-MS-Access. Originally it was to incorporate SmeQl, but I canceled that idea for various reasons, mostly because while writing the doc, I decided teaching the user both the tool and SmeQl was too much to absorb. -t''
----

Yes, other tools can do the necessary formatting, but it's usually more set-up steps. If you find yourself having to install, use, and run two tools together, that suggests they should become one (or at least extract and refactor the related features). 

[No, that means the tools should be part of the same toolset. Just because you need to use ProceduralProgramming and SQL queries together doesn't mean SQL queries should be capable of procedural behaviors. This has already been explained to you, multiple times, and you've seen plenty of examples... i.e. TutorialDee/RelProject can perform EXPORT or run WRITELN in a FOR loop; you do not need to install or run a separate OS process to use these export and imperative tools, and yet the RelationalAlgebra of TutorialDee cannot call EXPORT or WRITELN, and thus remains independent of (and uncorrupted by) those external tools. Maybe it would help if you start thinking of RelProject and SQL DBMSs as 'toolboxes' rather than as individual 'tools'.] 

Yes, but you had to introduce or rely on imperative operations to do it. There are better and cleaner linguistic and/or layering approaches to the same thing.

* ''How many '''more''' times do I have to point out that the loop was an illustration?  It's not a requirement.''
* How many more times do I have to ask for it's replacement?
* ''Myself and others have '''repeatedly''' mentioned EXPORT as one example.''

You have not demonstrated clearly what the "compromise costs" [downsides of bags] are. You say I'm ignorant of such costs. Yes, you're right, I am. You are not explicit enough. Your justification is HandWaving ("bad things happen, just trust me."). Reduce your explanations to a finer granularity and maybe you'll hit it. 

[These were explained in BagAtational, and have been explained elsewhere, and are well known in the literature. Your ignorance is not due to stupidity on your part, nor due to lack of detail available to you. Your ignorance exists because you are an ignoramus: you actually like to be ignorant. You are nobody's student. You don't even self-educate.] 

* Deflection. You are a poor technical documentor and you resist my suggestions for improving your writing because you are stuck in your ways.
* [Perhaps I am a poor technical documentor. That does not excuse your ignorance. After all, you are not my student.]
* It's a damn good thing.
* [Why is your ignorance a "damn good thing"?]
* No, it's a damned good thing that you are not my [cuss removed] teacher because you are annoying as hell and have a weird, roundabout writing style. I almost think you two are an AI experiment: you don't sound human.

And there's also the issue of being able to hook up to existing DB engines. If you follow their conventions, then hook-up is easier. 

[Sure, it's easier. RelProject will need to include some extra facilities for hooking up to SQL DBMSs, such as auto-numbering tables that potentially possess duplicates. On the other hand, why have a new DB engine if you're simply going to follow the conventions of existing ones? If 'easy implementation' is a priority, you should probably stick with SqLite or MySql or Oracle... because the 'easiest' implementation solution is to use an existing implementation.]

The existing DB engines are not the main problem (although I'd like to see DynamicRelational). SQL as a language is the main problem. I'm not bothered by bag and Null support.

----
OctoberTen