https://linoxide.com/linux-how-to/linux-fun-terminal-crazy-output/

Fun Stuff On The Web

This post is inspired by Austin and Paul who made great presentations about fun things you can do in *nix.  I did not know about any of these because apparently I live under a rock.

First off – cowsay.  You basically type in a sentence and then an ascii cow has a word bubble with your sentence. However, there are a ton of flags that you can use to change the cow and the representation and the sentence. And you an write your own cowsay file to further customize the cow. This being *nix, you can pipe it into another command – such as lolcats and print your cow with a gradient. I got all this info from Austin – thanks Austin.

screenshotThen Paul made a presentation on ponysay  and he made a recuse pony file. I am writing this blog post in a van from Phoenix to Tucson, and my friend next to me just said “there is nothing like a recusive pony!”  Indeed!.

So this prompted me to look at other fun things and nix easter eggs.

One I remembered from a long time ago is:

telnet towel.blinkenlights.nl

This is super old – but if you run it you basically watch an ascii version of starwars.

Figlet – this just gives you big unix ascii fonts- but can be fun to pipe.

Some other fun unix commands  are ascii aquarium  (repo) and ascii weatherspect – displaying fish and weather.

If you type :help 42 in vi you get

Screen Shot 2019-02-15 at 2.56.56 PM

rig creates random fake people, which actually may be helpful for testing

sl – creates a steam engine – my boys will love it

sudo apt-get install sl

Games in emacs: go to  /usr/share/emacs/*/lisp/play and ls. In emacs you can play one of these games with Meta-x so Meta-x life – plays game of life.

Screen Shot 2019-02-15 at 3.14.45 PM

While this was a fun diversion, there were a bunch of ‘funny’ unix commands that were in really bad taste and tone def. It makes me realize how some of computing culture needs to change to become more inclusive and empathetic.

Ports in Elm

I recently finished the basic Elm tutorial that Tenor introduced me to. The last assignment was on ports.   This to me seemed very strange. What is an elm port?  Why do we need it? It reminded me of this this Rust WASM feature request:

Request for library: mpsc channels library built on top of the postMessage API #163

This could allow wasm to talk to js via CSP style message passing, different wasm instances to talk to each other without sharing memory, the wasm instances might even be in different threads by using web workers.

So what are Elm ports? Ports are the way that javascript and Elm talk to each other.  In Elm ports, messages are one way. It is not like http or tcp/ip where there is a handshake. I can send a message from JS to Elm, and vice versa, but I do not confirm that the message was received.

It is a pub/sub pattern, where JS subscribes to a port that is populated in Elm, and then sends messages on a port that is then handled in elm.

I did not expect Elm to work this way, but its not that strange. Maybe I can use some concepts and apply them to RUST WASM.

WASM is Magic

A few weeks ago I stumbled upon wasm when I was

  1. Looking for Rust projects to contribute to
  2. Looking for a way to write smart contract that was not in javascript

I came upon wasm for both of these things. So what is WASM? WASM is web assembly. It is a binary format for web application.   You can write an application in Rust, or C, or any language that compiles into web assembly and then load it into a web page. The benefit is near native performance (and not having to use Javascript).   There are also some security benefits to the way that web assembly memory is allocated – the execution stack is separated from the WASM stack. Here is a whole medium piece  someone wrote on JS and memory if you are interested.

I found it very magical writing something in rust and then publishing it as a node package and calling functions in another js function.

There are two resources I looked at for web assembly. The web assembly book https://rustwasm.github.io/book and https://rustwasm.github.io/book/game-of-life/implementing.html. I recommend the second one because it has a really interesting example of canvas integration with WASM. There is also a podcast on this week in rust about WASM.   AND just because WASM was written for the web, and has ‘web’ in the name does not mean it could not be used for other things. It could be like a better type of virtual machine (think JVM) or something. Some other good links are the awesome rust and wasm page and wasm-bindgen examples.  I also heard about an open frameworks-ish library in WASM from Max D.

It also made me think about the way JS communicates to Non JS. So JS and WASM, or JS and elm (ports). I am not sure why this is interesting, but there is something about this sort of interop that I find worth thinking about it.

I still have some more work to do on my personal Rust WASM project, and am going to work with Marko on a Rust WASM version of his breakout speed analysis comparison. I was going to put a link to the heroku deploy – but because of the rust module this is not trivial. I think I need to write a heroku build pack or publish the package to npm. But here is a screen cap – its in the browser !

This is unrelated to WASM, but J has a great blog post about editing our git history. This is super helpful since I sometimes get lazy on personal projects and commit with unhelpful messages. My main git trick is bfg, when I want to remove something that I should not have committed…. It happens.

Capture the Flag

Today the coffee chat bot paired me with Mauricio.  When we met, he was working on pwnable with Cory.  I have never heard of capture the flag games until I started RC. These are games where you have to exploit some security bug in order to ‘capture the flag’.

It was super cool.   We worked on problem 10.  When you click on problem 10 the only instruction you receive is to ssh into a server.

Once you ssh into the server and run ls, you can see a c file, an executable, the flag, and intended solution. You only have permission to view the c file and execute the executable.  The only thing you can do is enter the right argument to the file.

The idea is that when you solve the problem and run the c program  with the correct arguments, then a bash command or script will display the flag. For example in problem 1, when you solve the puzzle the following code is executed:

system(“/bin/cat flag”);

The problems are about system exploitation, but really they are about operating systems and systems architecture. The way to solve the puzzles is by understanding how the computer works,  by inspecting the address locations of various functions, by understanding the stack and the heap, and by inspecting the machine code that is generated.

Problem 1 was about file systems.  There is clue that I misinterpreted as translating something from hex into ascii.

My misinterpretation lead me to read up on files in linux. I read the man page for read and attempted to understand the file descriptor. How do file descriptors really work?? On Linux fd refers to a process in  /proc/PID/fd/. I attempted to look in this directory but had no access. ps -ef also does not work – I dont have access.    The file descriptor also is an index to a table maintained by the kernel which is in index to another table.  This may be the solution to my problem.  But I sort of thought not, since I probably dont have permissions to any of these tables.

Then I methodically started reading the documentation on all the function calls like atoi. SPOILER ALERT.  I discovered that my initial intuition was correct but a bit off the mark – I wont tell you why but it should be obvious. I quickly was able to exploit the bug and capture the flag! What a rush!

Mauricio suggested picoCTF.  I also found this pretty exhaustive list. I got to say, CTF is a real thrill.

Music and Parsers

Photo note -I actually did see a prepared piano performance at Roulette about a decade ago when it was down in Soho. It was amazing! Now on to business.

Here is my first chess to music via Euterpea composition. Its experimental  lets leave it at that…

The Backstory….

A few Recursors and I  planned to write a chess notation parser (pgn) in Haskell last week.  I have been doing the Haskell exercises from this git repo, and I’ve been reading up on the basics from an online book – like WTF is foldable (its not that complex – just something that is reducible to less elements  like a tree or list). I could do some sort of brut force parser, but I wanted to learn something so I broke out my dragon book and dove in!

At this point it became evident that perhaps this project was less about Haskell and more about compilers. As it turned out I ended up not using Haskell at all,  but used Lex/Yacc  – which has fulfilled a decade(s) old wish.

What would a compiler look like if the language was pgn (chess notation) and we were going to compile it to musical notation to be executed by the Haskell library euterpea?What if we played a chess game as a musical composition?

This has been interesting to me for a long time since the 8×8 matrix of the chess board maps to an octave scale.  It is also interesting to me because you could also say that the rook, knight, bishop, king, queen map to the pentatonic scale.   It is also interesting to me because I am interested in transduction taking energy from one system and using it in another, so the energy in chess as powering a musical composition. It brings up for me all sorts of questions like what is the best mapping for this? Is there a best mapping. What will this sound like?  And so forth.

download (3)

Back to compilers! Generally, we think of a compiler as taking some source code and translating into machine code.   This goes like

  1. preprocessor – generate all code from macros and what not
  2. compile – translate the code to assembly
  3. assembler – translate the assembly to binary
  4. linker – link the libraries and other code

I am only interested in the compile piece  at this point – to turn one language into another (and not assembly in this case, but a protocol readable by Euterpea.  Although it is interesting to think about the other pieces. And in the case of compiler or code optimization we do want to look at the machine code or byte code or linking. (And maybe turn that into music too who knows)

To focus on the compiler -what I am interested in doing is

  1. creating a lexical analysis – that is a list of all the tokens in the language
  2. write  grammar that describes the language
  3. creating a parser – that builds the expression tree of how the tokens work together
  4. generate/emit the euterpea code/protocol

Lets get granular. “Here is the PGN format of the 29th game of the 1992 match played in Yugoslavia between Bobby Fischer and Boris Spassky” I took from wikipedia.

[Event "F/S Return Match"]
[Site "Belgrade, Serbia JUG"]
[Date "1992.11.04"]
[Round "29"]
[White "Fischer, Robert J."]
[Black "Spassky, Boris V."]
[Result "1/2-1/2"]

1. e4 e5 2. Nf3 Nc6 3. Bb5 a6 {This opening is called the Ruy Lopez.}
4. Ba4 Nf6 5. O-O Be7 6. Re1 b5 7. Bb3 d6 8. c3 O-O 9. h3 Nb8 10. d4 Nbd7
11. c4 c6 12. cxb5 axb5 13. Nc3 Bb7 14. Bg5 b4 15. Nb1 h6 16. Bh4 c5 17. dxe5
Nxe4 18. Bxe7 Qxe7 19. exd6 Qf6 20. Nbd2 Nxd6 21. Nc4 Nxc4 22. Bxc4 Nb6
23. Ne5 Rae8 24. Bxf7+ Rxf7 25. Nxf7 Rxe1+ 26. Qxe1 Kxf7 27. Qe3 Qg5 28. Qxg5
hxg5 29. b3 Ke6 30. a3 Kd6 31. axb4 cxb4 32. Ra5 Nd5 33. f3 Bc8 34. Kf2 Bf5
35. Ra7 g6 36. Ra6+ Kc5 37. Ke1 Nf4 38. g3 Nxh3 39. Kd2 Kb5 40. Rd6 Kc5 41. Ra6
Nf2 42. g4 Bd3 43. Re6 1/2-1/2

First I start with the lexical analysis.  This is what decides what and is what is not a token. I am going to throw out everything above the 1. So the game metadata really. I may want to change this in the future – but that is it.  I am also ignoring everything in {} again more metadata, and I am ignoring all spaces. I will keep tokens as Num. and the number 1-8,  abcdefghNBRKQ and /.

Next I attend to the parser or syntactical analysis. There are a bunch of parsers, and parser methodologies I could implement if I end up doing a Haskell implementation but for now I am using Lex/Yacc – which uses an LF parser.  Lex parses the tokens – it is like a bunch of regular expressions. Yacc defines the grammar and the emitter – which is fun because who does not love Backus-Naur Form??

The Lex/Yacc is not close to fully functional. The following  resources have been helpful. Sadly, I experienced a type error stripped all functionality out in an attempt to debug. Let this be a lesson. Commit early and often!

I should do some smart contract programming so I am just going to leave it half baked on github.  I have been playing around with some possible code generators (emitters) for Euterpea.   We can call this semantic analysis maybe (or maybe not). This is where we generate the musical notation for Euterpea. I had a great time doing this. I recorded the playback via quicktime audio recording because the code for saving Euterpea to wav was onerous. However, I think the background noise does add to the ambience. I just need to bump up the gain.

After this experiment, I learned that I should rethink the pitch mapping.  I would be curious to hear another chess game run through this. Will they all sound the same? If this is the case then my mapping definitely needs reworking!

Dipping my toe into compilers after decade(s) has been fun. I want to continue this discussion. Especially building the compiler in Haskell (not lex/yacc, building a recognizer in the lexical analysis (determining whether or not the input is in the language), parser deep dive, a symbol table deep dive, and of course, my favorite, code generation. But for now I am just happy to use the unix glue commands to generate a parser for pgn.

Making and Thinking

I always attributed this piece to sol lewitt. Imagine my surprise when this morning, after googling for an image I discovered this was by Lamont Young! C’est la vie.

This morning I finished reading Making by Tim Ingold.  I’m not sure if this is the best book to read by him, but I found it thought provoking as an exploration into anthropology and tool-thought (this is my neologism).  A lot has been made of the idea of embodied thought.  I ‘see’ that, or even embodied knowledge – like I cannot type my password unless I am at my keyboard. The locus of knowledge is in the movement of my fingers not in my mind.

How are tools created? How are objects created? How is art created?

Ingold is interested about the locus of knowledge in craft objects as they are used.  We cannot divide this into a hylomorphic discussion of form and matter.  Like a brick is clay plus a mould.  With Ingold it is a dialetic (he says dance) between the user, the tool, time, and decay.

He talks about the four A’s. Archaeology, which exists as the time dimension of a tool/artifact, as a tool changes over time in its ability to produce, with the user, some sort of understanding (knowledge).  Architecture, which is the physical location of tool, is anchored in tethered to the dimension of space. Anthropology is the ontological world view that the tool is embedded with in and which the tool expresses. Art is the on going unfolding or changing of the tool. The tool gives us one pieces of information today and another piece in the future that we cannot predict.

The most ephemeral, or rather, immaterial, of all tools, is software. It is also the tool that, perhaps unexpectedly, is most art like. We only understand or glean information from software when it is executed. And it often runs in ways we cannot predict.   Similar to the creation of a piece of pottery or a painting.

So why begin with the idea of drawing a line and following it – a piece of conceptual art similar in form to a computer algorithm? Ingold is obsessed with the hand, the physical creation of things. And for him everything goes back to drawing and the line. This is the basis for all creations, artifacts, objects, tools, buildings, etc.

He says “I am interested in drawing as a way of telling”  (p.125).   And then goes on to discuss all the drawings that do not tell but explain – like technical drawings or data visualizations (not drawings but you get the idea).  There is the drawing that is the expression of the line, of the idiosyncracy of the line, and the drawing where the line is commodified in order to communicate something else. The line is communication or the line services communication.

How does this fit with a computer, even with this wordpress text editor that I am using? There is no further way to commodify writing or gesture than to structure it as a series of binary digit.  Even Ingold wants his students to write by hand instead of with the computer. But is there a remnant of gesture in writing a computer program. Is there a statement that is pure communication instead of a conduit for communicating something else?

Software is gestural. It obliquely attempts to communicate its purpose and must be translated at least once if not more (in the case of virtual machines and bytecode) into machine language.  What would a physical relationship to writing software look like? A somatic programming language. Would this communicate something differently, more meaningful, more original about the universal experience?

Ingold’s idea about the co-creation of objects in time and space through the disciplines of archaeology, anthropology, art, and architecture are computational processes. Our gestures allow us only one slice in this computational process but in writing a program we can control the entire process.  What would this look like as an embodied practice and decommodified.

 

Terraform pt1 (really pt2): ipv4 to ipv6

I have been obsessed with terraform ever since I used ILP’s terraforms to create ilp connectors. I have mentioned this before.  Here is another book where terraform features prominently. This time it is Venus not Mars! I actually have a copy of this book, it was hard to find – ping me if you want to borrow it.

Why and what is Terraform…

There are a ton of options to automate, and ‘code-ify’ dev ops (get it code-codeify  hahaha). But most of the tools are about installing software on a server. I think docker sort of owns this. What terraform does is control the deployment of the servers or the platforms. Like you can use terraform to deploy k8 & docker.   Terraform is modular and declarative.   You could do server deployment using a shell script (I have done this a lot because I dont have to share my code).  However, this is harder to read, share, and maintain.

How to do things with Terraform

  1. Install terraform
  2. Create a directory where you want to create your terraform project. Git init it!
  3. Now – run terraform init
    1. if you do not do this then terraform will not be able to find a “provider”
  4. Create a main.tf file. This is your terraform file. You can put all kinds of stuff in here and get fancy. But you need to specify a provider. In this case I am using aws:
provider "aws" {

region = "us-east-1"

}

5. Next run terraform plan. This shows you what you are going to deploy.   If you want to save the plan so many people can deploy the same plan then just use the –out command.

6. Finally run terraform apply  – (or terraform apply “planyoumade”)

7.  You will see a terraform.tfstate file

8.  fin!

What did I do in my terraform code?

  1. I have  two files:.variable and main. I could also put this in other files to make it more modular, but KISS (keep it simple stupid).
  2.  I try and offload as many variables to environment variables as possible (see the your.env)
  3. I created  a free ec2 machine using the ubuntu 14.04 ami – and then a vpc, routing tables, security settings and I think that is it
  4. terraform apply

This was fun, I never thought I would be like one of those dev ops guys with a beeper – but maybe I am one of those guys. Maybe I should get a beeper on ebay and wear it as an anachronistic accessory.

I had a lot of issues around creating route tables (route tables vs default route tables.  Basically when you create a route table you have to recreate all the routes and this causes terraform to freak out.  There is a scary warning on the hashicorp site about this.  I spent a fair amount of time googling to figure out what the heck was going on.  At one point it seemed that I would have to use the aws cli instead of terraform and sank into the trough of dispair.

Then I realized I did not git ignore my .terraform file. So I just blew away the whole repo, recreated it ran terraform apply and it worked!  Magic

The other magic was reverse tunneling over two machines so that we can use jupyter notebook. I accomplished this though a somewhat hacky local-exec resource. The better solution would be to create a script – or maybe dockerify nginx.

Be careful! You can quickly use up your internet gateways – I dont think my delete works properly.

One thing I would like to leave you with is the notion of Egress only gateways – which is a internet gateway optimized for ipv6 from the VPC to the internet.  To quote Amazon:

An egress-only Internet gateway is a horizontally scaled, redundant, and highly available VPC component that allows outbound communication over IPv6 from instances in your VPC to the Internet, and prevents the Internet from initiating an IPv6 connection with your instances.

I am not sure how this would work with my tunneling though so I would have to do more debugging.  Which I am not going to do right now. So this is perhaps an open issue that someone can use to contribute to this repo 😉

The readme is a marginally better than this blog post.  So if you actually want to use this to create a ipv4 to ipv6 gateway and reverse tunnel so you can run jupyter notebooks take a look.

IPv6 vs IPv4

I am debugging my terraform script to create an IPv6 gateway and tunnel. But what is IPv6? AND what is IPv4?   T

IPv4 has 32 bit addresses so there are 2^32 available ip addresses. All of these are almost used up, so a few years ago IPv6 was introduce – 128 bit addresses 2^128 addresses are available. Obviously that is a lot more.  The networks run parallel and you need a gateway that turns your ipv4 packet into IPv6 (I think) I could be wrong here.  The packet structure is different as is the address name space.

When we are talking about IPv4 and IPv6 gateways we are talking about sockets.

Lets take a look at python sockets!

serversocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
serversocket.bind((socket.gethostname(), 80))
serversocket.listen(5)

This opens a socket on port 80.AF_INET refers to IPv4. (like a http server) If I wanted to open a socket on IPv6 I would use socket.AF_INET6.

In this case I bind the socket to an address like localhost or 127.0.0.1 – or a 128 bit IPv6 address.

AF_INET returns a 2 tuple – and address and port. AF_INET6 returns a 4 tuple: address, port, flow info, scope id.    Support for flow info and scope id is part of the protocol that makes these two incompatible.

What about the rest of the packet? I think I need to do some socket programming to truly gain intuition.

 

ELM workshop

Yesterday Tenor gave an Elm workshop at Recurse. It was probably one of the best programming workshops I have attended. At recuse there are a bunch of elm-ers (is that what you call elm programmers) and so I hope to find someone to pair with.

Many years ago I looked at elm to use for the front end to my company site Print all Over Me. We ended up using react because the ecosystem was too young. This was like 2015. Fast forward 2018 we moved everything back to shopify  so it really did not even matter! But what I did learn is that although I can program in React, I don’t love it. There is too much boilerplate and I think user defined tags (components) are too  OOD. I am a functional kinda gal.

The workshop Tenor gave is on github and I highly recommend it. Basically you just fill in the TODOs in the different sections and learn elm. I asked Tenor if there was something like this for Haskell, he said no and I cried (inside).   I love Haskell because it is a programming language like a formal language.  I feel like I am writing grammars when I write Haskell (which I do very poorly).   On a site note it would be interesting to look at grammars and Sanskrit – since this is a language generated by a grammar.  Grammars and generally descriptive not generative.

Elm I like, it is like Haskell, and like Scheme (my first true love). Functional, modular, not a lot of code. Sadly there is no WASM for elm, but apparently it is in the works.  There is a web3js elm package so for any future web dev for Prayer coin and I going to use elm.  Life is too short for boilerplate.

I’m going to take this opportunity to just dump a few things down that have come across my ‘docket’ and that I have found interesting.

Time Spirals  – great visualizations made by a Recurser shared on zulip

Compromise Cool and ml5js – Machine Learning on the web

IOS and Rust  Even if I never build an IOS app again this helped me understand how rust plugs into different code ecosystems

Racket  a language for making domain specific languages – not sure what I will use this for – if for anything – ever – but it seems cool!

I am almost done with the book on psychoanalysis technique. Dream is the royal road to the unconscious. What does that mean?  Maybe it is the most direct route? The Route with the cleanest path? The most travelled path?  Royal in what way? According to Wikipedia, the first royal road was in Persia and was rebuilt from a perhaps less royal road by Darius the Great.  It was made to aid in communication. This is perhaps different from logistics or deployment as the Interstate (and Autobahn) were made.  But then perhaps our technologically complex world is about converging communication with action (logistics).  The royal road is a easy way to communicate. It is the SMS to the unconscious.  Its a great metaphor.

 

The balance dap

I finally finished debugging the ERC20 balance coin front end. This was just a proof of concept experiment to create a ERC20 contract and create a web interface (dapp).

Balance coin is not really a fully fleshed out idea but a vehicle for me to build a dapp.  Here is the dapp and the code. It needs cleaning and linting, beautification (because I really dont know material-ui) and the meditate and social media buttons need work BUT you can send balance.

That said – I was kinda thinking about sweatcoin and how it would be interesting to have a coin that gained value the more balance you had (ie mindfulness/mental clarity/ that kind of thing). To that extent, you can add to your own supply of balance (and total supply balance) by doing mindful things such as meditation, and lose balance by doing not so mindful things (e.g., dopamine hits on social media).  You can also transfer balance coins to other people.  It developing the dapp I discovered a bunch of ways the contract was unstable – so I should probably update that – but I am moving on to NFTs since that is the crux of the prayer blockchain.

This all runs on the ropsten test network as I mentioned in a previous blog.  The code needs to be formated and linted but it is pretty simple so anyone can take a look and see how to build their own dapp. One of the things I would like to do now is a code generator to create dapps.

My main issue was that I was using web3 1.0.0-beta.41 and there was a bug and I had to downgrade to 1.0.0-beta.37.  This took about 5 hours to figure out, but I did become very proficient at using the truffle console to examine tokens!