YetAnotherForum
Welcome Guest Search | Active Topics | Log In | Register

Squirrel as a standalone general-purpose scripting language
pfalcon
#1 Posted : Wednesday, July 31, 2013 8:23:18 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 1/9/2013(UTC)
Posts: 59

Thanks: 1 times
Was thanked: 3 time(s) in 3 post(s)
Squirrel had started as an embeddable scripting language, and that's how many people use it and see as an alternative to Lua. But Squirrel is a well thought, designed and engineered language which can be serve as a standalone general-purpose scripting language if some issues are resolved. I'd like to dedicate this topic to discussion of how to achieve that. But first, let's consider why someone would want to use Squirrel as such in the first place, if there're some many well-established alternative exists, like Perl, Python, Ruby, JavaScript (Node.js), etc.

So, the answer is the Squirrel size and "lean and mean" nature. You can get complete interpreter under 200Kb, and can optimize the size further. It also doesn't have big standard library, actually, it doesn't have any - just standard functions and methods, so simple applications can stay simple and small. Interpreter size is definitely not a problem on a desktop, but nowadays, embedded systems become more and more of a commodity, and people want to program them comfortably. I'd really like to use Squirrel on an Arduino, but unfortunately that's not achievable. But a wireless router with 2MB flash seems like good target for it. And actually, I'd hope Squirrel to be usable on more pumpy than Arduino MCU boards with Cortex-M CPU and ~256KB of Flash, as already exemplified by Electric Imp. Then, the thought occurs - if Squirrel is comfortable to program and fits on small system, why not scale it up and use on desktop and server too? Well, that's the motivation for turning it into a general-purpose language.

So, what requirements should be followed why going in that direction, what Squirrels lacks, what should be added, what issues addressed? Here's rough list:

1. Whatever being changed with Squirrel, it should stay small, simple, and embeddable. Those are core feature of Squirrel which got us interested in it at all, so we should not compromise on them. If we add to many features to it, it becomes bloated like a typical "desktop" scripting language, but there're bunch of them already for immediate use.

2. There should be good interactive interpreter, because there's no better way to learn language than to fire up interpreter and try something interactively.

3. Deviations from conventional "median" scripting languages syntax/semantics should be massaged to allow more conventional approach as alternative. E.g., RFC to make "<-" optional.

4. Quite minimal standard library is what makes Squirrels lean (what is called "standard library" in Squirrel in other language would be "builtin types and functions"), but of course is the biggest obstacle to becoming a useful general-purpose language. And yet it should stay the same. That means that all extra functionality should go into separate, optional modules. That way, only needed set of modules (including empty) can be deployed for particular usage.

5. There's good foundation of "streams" base class, but builtin file support is quite lacking, there should be possible to trivial file processing out of the box (like, iterate over each line in text file, do something to a line, write to another file).

6. To support extensibility, there should be genericmodules support, and that's what Squirrel currently lacks, and the first feature to address.

7. There should be support for Squirrel source file as modules, compiled bytecode, or dynamically loadable compiled extension. All should be supported transparently, so during development and on desktop source modules can be used, for deployment to constrained environment - bytecode, and to optimize performance - C extension. 'import("foo")' should try to load each variant in suitable order.

8. There should be package manager with networking capabilities and central repository(ies) of available modules, like available for most other popular modern language (heck, even Lua has one!).

9. One of the first modules to develop should be FFI (Foreign Function Interface). This will allow to interface with vast selection of existing C/C++ libs without needs to write C extensions.

...
pfalcon
#2 Posted : Sunday, August 4, 2013 12:53:08 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 1/9/2013(UTC)
Posts: 59

Thanks: 1 times
Was thanked: 3 time(s) in 3 post(s)
Posted RFC for (elaborating) modules support: http://forum.squirrel-la...aspx?g=posts&t=3122
pfalcon
#3 Posted : Friday, August 9, 2013 9:15:54 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 1/9/2013(UTC)
Posts: 59

Thanks: 1 times
Was thanked: 3 time(s) in 3 post(s)
Added point on elaborating builtin file/stream operations - it's not possible to do simple text file processing using Squirrel currently.
pfalcon
#4 Posted : Friday, August 9, 2013 9:22:09 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 1/9/2013(UTC)
Posts: 59

Thanks: 1 times
Was thanked: 3 time(s) in 3 post(s)
I also looked into improving interactive interpreter (II) - ideally, it would work as Python, automagically printing result of any expression if it's non-null. But well, that's not easily possible with Squirrel - the way II works, it wraps each input statement in function, and executes it. There's not possible to execute just bytecode statement - only function. And there's no implicit return of last expression in a function, so behavior above is not possible. Then the best thing I could do is to make II print intro/help text at start: https://github.com/pfalc...774ad88cc80e00590927f93

Quote:

Enter statements, each will be executed in a standone function (that means local variables won't work between two statements). To print result of an expression, prefix it with '='. Use quit() or Ctrl+C to quit.


I also made sure that non-interactive interpreter returns with non-zero code in case of runtime error.
pfalcon
#5 Posted : Monday, August 12, 2013 9:54:27 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 1/9/2013(UTC)
Posts: 59

Thanks: 1 times
Was thanked: 3 time(s) in 3 post(s)
Ok, stuff shapes out pretty well. I designated https://github.com/pfalcon/squirrel-modules as the main repository for "General-Purpose Squirrel", it pulls patched Squirrel core as a submodule. It can be built with simple "make" and installed under your home dir with "make install". The latter will install "sqrl" interpreter, few modules and startup module (yes, there's now support for auto-importing a module with designated name on interpreter startup, that's how high-level import framework implemented in Squirrel itself is initialized for example). Everything is described in more detail in the README on the page above .

Well, it was 2-week sprint and IMHO I was able to bootstrap "general-purpose" effort quite well - all results of pondering about that for several months before. There's definitely a lot more to do, but I guess it's time to put it back into my round-robin queue, and switch to gazillion of other projects I have there, while pondering what to do next. One thing I definitely would like is to receive more feedback. But I guess it's fair to say that Alberto is not interested in changing language to be general-purpose, so prerequisites changes to the language core are unlikely to be merged, so this all looks like a fork. So, need to think about name, and how to organize it all. Again, suggestion and discussion is welcome.
pfalcon
#6 Posted : Wednesday, August 14, 2013 10:56:34 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 1/9/2013(UTC)
Posts: 59

Thanks: 1 times
Was thanked: 3 time(s) in 3 post(s)
Turned out there was missing header file to implement this proposal http://forum.squirrel-la...sts&m=6963#post6963 . Of course, this header needs to be autogenerated from existing API method table definitions, and of course, that should be done with Squirrel script, but Squirrel lacked string I/O on files (well, I implemented readline() before). That's now fixed to provide .read() and .write() methods, and the script is written: https://github.com/pfalc...le/make-compat-syms.nut . That's the first actually useful Squirrel script I wrote ;-).

Btw, some pondering makes it clear why string I/O on files is not supported in core Squirrel: because char type may vary, and then we need to deal with converting internal char encoding to external and vice versa. And that's boring and outside the scope of nice clean language core. But how General-Purpose Squirrel? Very easily to, by enforcing well-known invariant - UTF-8 is *the* encoding. That means that SQChar is always char for G-P Squirrel. Of course, you can write your own module supporting UTF-16 or whatever char type.
pfalcon
#7 Posted : Friday, August 23, 2013 8:05:20 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 1/9/2013(UTC)
Posts: 59

Thanks: 1 times
Was thanked: 3 time(s) in 3 post(s)
Changed arguments passed to a script - vargv[0] is now script's own name. This is similar to change in Sqilu and of course like most other (unixy) scripting languages have it.

Other question is how to detect if a script file is run standalone, or was imported from another script. Python has following idiom:

Code:
if __name__ == "__main__":

I consider adding something similar...
pfalcon
#8 Posted : Monday, September 16, 2013 9:15:47 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 1/9/2013(UTC)
Posts: 59

Thanks: 1 times
Was thanked: 3 time(s) in 3 post(s)
As a small update, popen module was implemented: https://github.com/pfalc...dules/tree/master/popen . This actually should be merged with fdopen() into something like "osstream" module. Allowing to wrap user-defined class into a (C++-level) stream interface would make stream subsystem pretty complete for general-purpose usage.

pfalcon
#9 Posted : Tuesday, April 22, 2014 8:28:45 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 1/9/2013(UTC)
Posts: 59

Thanks: 1 times
Was thanked: 3 time(s) in 3 post(s)
There were no updates in a while, so here's what happened. I took couple of months to consider if I really want to go forward (and trying to find other interested people in the meantime). By the end of November I decided that it's worth to continue, and I was almost ready to restart hacking, when I suddenly heard about "MicroPython" project. From the discussion above and in other topics I opened it's clear that I tried to make Squirrel more similar to Python, and actually I thought a lot if instead of hacking on Squirrel I tried to slim down Python. Emergence of clean-room unbloated implementation of Python resolved any uncertainty - I since then fiercely hacking on MicroPython ;-).
1 user thanked pfalcon for this useful post.
PhillipT on 4/25/2014(UTC)
PhillipT
#10 Posted : Friday, April 25, 2014 11:58:23 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 4/25/2014(UTC)
Posts: 1
Location: USA

Thanks: 1 times
Was thanked: 0 time(s) in 0 post(s)
PFalcon - sorry to hear that, it was actually recently stumbling across your proposals and SquiLu that made me take another look look at Squirrel. MicroPython is awesome and a good project and glad you found something to hack on. I think Squirrel is a strong match for the scripted simulation problem set I work with, so will stick with it (really like the language design). I have friends who are very excited about MicroPython for embedded so can understand why you are keen. Thanks again for the proposals and energy and good luck with your new focus.

pfalcon
#11 Posted : Sunday, April 27, 2014 10:18:06 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 1/9/2013(UTC)
Posts: 59

Thanks: 1 times
Was thanked: 3 time(s) in 3 post(s)
Thanks for feedback! I always considered "general-purpose Squirrel" idea as a good chance to start afresh, to create lean and mean general purpose language without superfluous parts and duplications. Of course, you first need to design it in such way, which is not easy ;-). With Python, you get extensive API, but you need to implement it, and then there're gaps or overlaps in it which steal a bit of satisfaction. MicroPython also has its traits, for example, it uses GC - which is the choice to get minimal memory overhead, but scales worse and worse as heap size increases. So, you can't have it all perfect, and I still think that Squirrel is an excellent language, both as embedded and potentially, as general-purpose one. For me, my good affinity with Python and good understanding with MicroPython's author made the decision, but I'm sure hope Squirrel will keep its loyal users and get more!
MicioMax
#12 Posted : Saturday, September 16, 2017 9:31:24 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 9/16/2017(UTC)
Posts: 1

Thanks: 0 times
Was thanked: 0 time(s) in 0 post(s)
Hi pfalcon,

I stumbled today on this thread, looking for squirrel info.
I'd like to try it on my Fishino 32 bit boards, so I'm wondering if you abandoned your "general purpose squirrel" project or it's still active.

Ciao

Massimo
Users browsing this topic
Guest (5)
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

Clean Slate theme by Jaben Cargman (Tiny Gecko)
Powered by YAF 1.9.4 | YAF © 2003-2010, Yet Another Forum.NET
This page was generated in 0.132 seconds.