Every API is an invented language

When saying that one of the reasons to use languages like Lisp, Ruby or Smalltalk is that they allow you to create your own domain specific languages, some people argue against it. They say it would create unmaintainable code. 

But a library IS a form of invented language.
For example,

'''If you created a DSL, other programmers would have to learn about your language:'''

But does that mean if you created a library, or a function, other programmers will never have to learn about your function?

Why is this API considered ok

 x = new SAXXMLDocument("foo.xml");
 while(!x.isEndOfDoc()){
 ..node = x.getNextNode();
 ..switch(node.name){
 ....case DATA_SET:
 ...... //do things with data set node
 ...... in_data_set = true
 ....case DATA:
 ...... if(in_data_set) then
 ........ // do things with data node inside data_set node
 ...... else
 ........ // do things with outside data node
 ..}
 }
 
While this invented language is so hard to use

 (readXMLDocument "foo.xml"
 ..(DATA_SET (data_set_node)
 .... do: (;; do things with ROOT_DOC)
 .... (DATA (data_node)
 ...... do: (;; do things with DATA node inside data set node)))
 ..(DATA (data_node)
 .... do: (;; do things with outside DATA node)))

To use the API library, you have to know what parameter to call, what return value to expect, what to do next with the returned value. There is no harder to write all that code set up to use a library than to use an invented language for a domain.

-- PisinBootvong

----

While I agree with Pisin, I feel some need to clarify what I consider the valid extent of the claim: I suggest it is '''protocols''' and '''frameworks''' that really transform an API into a language, and that the degree to which ApiIsLanguage stands on a spectrum. OTOH, I would agree that ''with very few exceptions, every significant API ever written is or becomes a framework''. See FrameworkIsLanguage for details.

----
''There is no harder to write all that code set up to use a library than to use an invented language for a domain.''

You don't have to learn new syntax or idioms to use an API. APIs aren't language, they're vocabulary.

''I see a misunderstanding here. The point is not about syntax, but about learning the rules of the business logic represented by the API. Granted, that ''may'' be easier if it is represented on the syntax level, than if it is represented by an own language. But that's not the point.''

''On the other hand: The inherent complexity of the 'language' used to represent the business logic may easily drown the complexity of the API. It just depends on your API - and how fluent you (read: MaintenanceProgrammer) are in the language-definition-capabilities of the source-language. ''

''-- GunnarZarncke''

Sorry, I don't understand what you're saying. Knowing "what parameter to call, what return value to expect, what to do next with the returned value" is not the same as learning a new language. The syntax and idioms are already known. All that is learned is vocabulary. APIs don't define new languages, they add words and usage rules to existing languages.

''On the programming language level you are right. But that's not the point made by the original poster. I think, he claims, that the API itself - and its use - constitute some kind of language (or protocol) which you have to learn.''

''Example: Consider a memory-API with malloc, free, realloc. Granted, these are simple functions. But you cannot call them in any order with any parameters. Some sequences are OK, some are not. You must learn and understand which are which. The calling-protocol is some kind of grammar or super-language, which you will not get around.''

''And now comes the insight: If you can create a simple language, that matches the implicit super-language of your API, it will actually simplify understand and writing code, not complicate it.''

''-- GunnarZarncke''

Yeah, that is exactly the point I'm trying to show. You never call only one function call for a library, but you have to know other requirement/protocol of such library as well. The order of malloc and free is simple example of that. Other example is, Strut, Database Query, GUI event dispatching. All of that, there are predefined step of thing you have to do, you have remember all that steps. There is no different from remember those steps and language syntax. --PisinBootvong

But it ''is'' different from learning a new language. The syntax is already known. A new language might be simpler than an API, but that doesn't make an API a new language. (And a new language really isn't simpler than an API in my experience. I often call just one function in a library, and I don't have to learn new syntax to do it.)

''But being able to call it doesn't help you if you don't know, what it does, what parameter it expects, where and how you have to create these parameters, which methods you have to call afterwards. The longer I think about it, the more I think, that it is good to have an own language for these dependencies. It a) forces the user to learn exactly this and b) provides syntax cues to remember it easier. But one has to be careful not to overdo it and reuse language for alike tasks.'' -- GunnarZarncke

But I know that "call" is something I can do on the API. I know "parameters" are something it might expect. I know it has "methods". I know all of this because I know the language it is expressed in. If I have to learn a new language, I have to learn those dependencies plus syntax. Learning a new language doesn't reduce the need to learn the dependencies, it just adds the cost of learning a new language as well.

''Your last sentence is the key. You are right IF the language is not appropriate. This may simply be true because it is too hard to learn, because you are not used to do so. But the languages mentioned in the first paragraph of this page are usually easy to learn.''

The languages mentioned in the first paragraph are not the new languages we're talking about. They are the languages the new languages would be written in. It doesn't matter what the new language is written in. Knowing that won't help the user learn the new language.

  [...] Lisp, Ruby or Smalltalk [...] allow you to create your own '''domain specific languages'''.

''I think, that expressing domain specific dependencies in a domain specific way can help understanding just these dependencies.''

But domain specific dependencies can be expressed in a domain specific way without introducing a new language. There are cases where one language is more appropriate to a class of problems than another language, but general purpose languages like Lisp, Ruby and Smalltalk are fine for almost all problem domains.

----

I agree with the basic point the page is making but what's not mentioned is that many API's, assuming they are popular enough for someone to write wrappers, are available in different languages. If, for example, I'm familiar with an API or large library (e.g. Win32, Gnome) then I can use that API from many languages. I do this, for example, when I use ActiveMIL (an image processing library) from Ruby for quick prototyping and then writing the final product in C++. -- AndrewQueisser

----

My problem with these mini-languages is that it is unclear how they interact with the rest of the "host language". A mini-language may give me a place to put some number. Is it ok for me to put a conditional in there? If so, what is in scope so I can evaluate the conditional? If I am working with two mini-languages, can I intermingle them? What are the limitations for that? These questions always come up. When writing code directly in the language itself, there is no question about these important issues. -- MichaelSparks

''Best practice of building a mini-language, IMHO, is to build it on top of working function call. Therefore, if the provided syntax cannot cope with a special case, one can always go back and use the API directly. In big API, this is true, it's build on top of many small function, API is usually wrapper over series of function calls. -- PisinBootvong''

----

''Good points. And I have another one: The language for APIs evolved. In the beginning there were machine traps, then simple method calls. Now some OSes have OOP apis (if you consider the Java VM or .NET an OS, you have multiple languages, that share an OOP base). I wonder if higher level languages will slowly become more commonplace such that we will be able to use mini-language APIs.''

----

I agree that API can be used in another language, while DSL can't. But to be able to use those API in another language, all you can do now is to express them in term of C function call. And then wrap it back to higher level again. 

At this point, I may as well say that DSL can be used across language too, but you have to translate such syntax back to its API representation, then make a wrapper for target language using DSL again. This looks almost like the same process for using an API in another language. Except that this only work with language that support defining syntax though; which is about the same, you cannot make OO wrapper of language that lacks OO and make it still looks like original API.

The process of making a wrapper for one API across a programming language would look like this.

 +--------------+  +--------------+
 |  Lang A API  |  |  Lang B API  |
 +--------------+  +--------------+
        |                  |
        |                  |
 +------------------------------+
 |             C API            |
 +------------------------------+

However, good library tends to be made of many little function calls (so that you can have flexibility in edge case). So it would look like this

 +--------------+  +--------------+
 |  Lang A API  |  |  Lang B API  |
 +--------------+  +--------------+
        |                  |
 +--------------+  +--------------+
 | Lang A Funcs |  | Lang B Funcs |
 +--------------+  +--------------+
        |                  |
 +------------------------------+
 |             C API            |
 +------------------------------+

The process of making a mini language wrapper to other programming language would look like this.

 +--------------+  +--------------+
 | Lang A Syntax|  | Lang B Syntax| ;; only if Lang B support ability to create mini language.
 +--------------+  +--------------+
        |                  |
 +--------------+  +--------------+
 |  Lang A API  |  |  Lang B API  |
 +--------------+  +--------------+
        |                  |
 +--------------+  +--------------+
 | Lang A Funcs |  | Lang B Funcs |
 +--------------+  +--------------+
        |                  |
 +------------------------------+
 |             C API            |
 +------------------------------+

About the ability to comprehend mini-language or ability to intermingle two mini-language, I thinks it's about design. Not that every API can be use together (May be now I'm thinking of Framework, not API. Many Framework are designed as if it will be the only framework in an application, But there is not clear distinction between Framework and API anyway).

Both API and mini-language struggle the same reason for portability, the binary interface of function call is still stuck at C! OO API have a hard time porting to other language, FP API have even harder time (because they not only need object reference, they also need closure) porting to other language. Unless we standardize those language feature, higher level library will never be as useful across language. This problem is one big source of reinventing the wheel, if we can use many OO or FP API without minimal wrapper code. Many XXX port in Perl, Java, C#, Lisp, Ruby would not occur this much.

-- PisinBootvong

----
See DomainSpecificLanguage, ExtensibleProgrammingLanguage, CrossToolTypeAndObjectSharing