Cardano Plutus development environment: Jan 2022

A lot of people are having problems setting-up a dev environment with Plutus PAB and Plutus Playground. In this article, I am going to summarize the method I use to install everything and get it working on an Ubuntu virtual machine with 8Mb of available RAM.

WARNING: this article will age relatively quickly and is only verified for the time of writing (January, 2022). This represents the current canonical build for installing a working instance of Plutus PAB and Plutus Playground. There are a number of reasons that people are having so many problems:

• Using out-of-date instructions
  There are still a bunch of instructions from previous cohorts of the Plutus Pioneer Program (PPP) that are no longer applicable. The instructions in this article are only good for the 3rd Cohort (Jan-Mar, 2022) of the PPP. Things can and probably WILL change.
  
• Using the wrong git repository
  There are some different git repositories around (plutus, plutus-core, etc.) but the current repo that you should be using at this point is the plutus-apps repo: https://github.com/input-output-hk/plutus-apps. Accept no substitutes!
  
• Missing important setup steps
  In particular, the Nix build and package management system is quite complex and there are a few pitfalls to watch out for. The more you read about and understand how Nix works, the fewer problems you should experience.

I : Install the toolchain: Git & Nix

You will need to have git installed on your system to checkout the code from the repository. On most Linux systems, git will already be preinstalled. If not, you can install using your Linux package manager. On Ubuntu, that would be a command on the terminal something like:

# update your package list
sudo apt update
# install git curl
sudo apt install git curl
# clone the plutus repo
git clone https://github.com/input-output-hk/plutus-apps

Next you will need to install and configure Nix as a multiuser installation (recommended):

sh <(curl -L https://nixos.org/nix/install) --daemon

WHAT IF this gives me an error?

NOTE: This command can fail (it does on my Ubuntu install) and IF that happens you can use the fail-safe method to run the script for a multiuser installation (recommended):

# download the install scriptcurl -L https://nixos.org/nix/install > nix-install.sh
# make it an executable script
chmod 755 ./nix-install.sh
# run the script
./nix-install.sh --daemon

DO: run these commands as a regular system user.
DONT: run them as root or with sudo — Nix will invoke sudo when it needs to.

WHAT will it do?

1. Create a system group called nixbld
2. Create 32 user accounts (nixbld1 - nixbld32) and assign them to the nixbld group
3. Backup your /etc/bash.bashrc file to /etc/bash.bashrc.backup-before-nix
4. Add a new /etc/bash.bashrc file with references to /etc/profile.d/nix.sh
5. Setup and start the nix-daemon --daemon background process
6. Create a TCP/IP socket: /nix/var/nix/daemon-socket/socket

WHAT IF I mess it all up and have to start again cleanly?

sudo rm -rf /etc/profile/nix.sh /etc/nix /nix ~root/.nix-profile ~root/.nix-defexpr ~root/.nix-channels ~/.nix-profile ~/.nix-defexpr ~/.nix-channels 

# If you are on Linux with systemd, you will need to run: 
sudo systemctl stop nix-daemon.socket 
sudo systemctl stop nix-daemon.service 
sudo systemctl disable nix-daemon.socket 
sudo systemctl disable nix-daemon.service 
sudo systemctl daemon-reload

# restore your /etc/bash.bashrc profile from backup
sudo mv /etc/bash.bashrc.backup-before-nix /etc/bash.bashrc

HOW DO I KNOW that the service and sockets are running properly?

If you are running a Linux distro that uses systemd (like Ubuntu), you can control your processes using the systemctl command.

VIEW the status of your daemon service

sudo systemctl status nix-daemon.service

You should see OUTPUT like:

● nix-daemon.service - Nix Daemon
Loaded: loaded (/etc/systemd/system/nix-daemon.service; linked; vendor preset: enabled)
Active: active (running) since Tue 2022-01-11 23:46:52 UTC; 11h ago
TriggeredBy: ● nix-daemon.socket
Main PID: 1976632 (nix-daemon)
Tasks: 5 (limit: 9458)
Memory: 1.0G
CPU: 8min 12.334s
CGroup: /system.slice/nix-daemon.service
└─1976632 nix-daemon --daemon

Jan 12 00:35:21 localhost nix-daemon[1976632]: accepted connection from pid 2149020, user nyk
Jan 12 00:35:22 localhost nix-daemon[1976632]: accepted connection from pid 2149020, user nyk

VIEW the status of your socket

sudo systemctl status nix-daemon.socket

You should see OUTPUT like:

● nix-daemon.socket - Nix Daemon Socket
Loaded: loaded (/etc/systemd/system/nix-daemon.socket; enabled; vendor preset: enabled)
Active: active (running) since Tue 2022-01-11 23:43:00 UTC; 11h ago
Triggers: ● nix-daemon.service
Listen: /nix/var/nix/daemon-socket/socket (Stream)
CGroup: /system.slice/nix-daemon.socket

Jan 11 23:43:00 localhost systemd[1]: Listening on Nix Daemon Socket.

II – Build the project

First, you must configure Nix to use the IOHK nix cache, otherwise the build will take many hours as without the cache it will compile the GHC libraries from source. You can do this in either /etc/nix/nix.conf or in ~/.config/nix/nix.conf. Add these two lines (two key/values) to the nix.conf file:

substituters = https://hydra.iohk.io https://iohk.cachix.org https://cache.nixos.org/
trusted-public-keys = hydra.iohk.io:f/Ea+s+dFdN+3Y/G+FDgSq+a5NEWhJGzdjvKNGv0/EQ= iohk.cachix.org-1:DpRUyj7h7V830dp/i6Nti+NEO2/nhblbov/8MW7Rqoo= cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=

Now go to the plutus-apps root directory and start the build by entering the Nix shell:

cd ~/plutus-apps
nix-shell

It will start to output trace messages to your terminal as it builds the dependencies for the project. If the messages say that it is compiling GHC and you see no messages about copying files from hydra.iohk.io, iohk.cachix.org, and cache.nixos.org, then exit the shell (ctl-c). Check your nix.conf file to make sure everything is correctly setup and maybe try to restart the nix-daemon process. Then try again.

III – Start the Plutus Playground Server & Client

From within the nix shell, you will move to the plutus-playground-client directory to start the server:

nix-shell #if you had left the nix shell
cd ./plutus-playground-client
plutus-playground-server

The service will start, running in the foreground, which means that if you close the terminal, the service process will also terminate (i.e. it does not run as a daemon process in the background). To build and start the client service (built with Purescript), you must start another terminal with another nix-shell:

cd ~/plutus-apps
nix-shell
cd ./plutus-playground-client
npm start

Once the compilation of purescript has completed and the webpack has run, you should be able to open up the web interface in your browser at https://localhost:8009/. That’s it!

Toptal – the top 3%

One of the challenges of working freelance is keeping your project pipeline balanced, so that clients aren’t waiting too long for your availability and you are not left with gaps between projects. This usually means having to split your working day between ongoing business development initiatives and your actual paid project work.

Freelance networks like Toptal can potentially help keep the project pipeline primed so that you can focus more on your primary core skills. You know, the ones that are actually billable. A web application programmer is often not only programmer, but must serve many different roles: operations (DevOps), first-level testing and Q/A, database administrator, incidental graphic designer (not typically the strongest card in our deck), etc. When you are freelance you have doubled the number of roles.

That is why Toptal Web Development Network  could be an attractive solution for keeping that project pipeline healthy; freeing yourself to focus on improving the ongoing quality of your billable work. More billable hours means more revenue, or more time to spend with your family. Just as you might hire an accountant to take care of your book keeping, why not join a network to free you from some of the burden of business development. It just seems to make sense.

Disable ICMP ping replies (stealth mode) with sysconfig.

Tags

icmp, security, sysconfig

Edit /etc/sysctl.conf and add:

net.ipv4.icmp_echo_ignore_all = 1

Enable the setting with:

sysctl -p

Check the setting with:

cat /proc/sys/net/ipv4/icmp_echo_ignore_all

Why raising the MySQL open_files_limit may not work and how to fix it.

If you are running MySQL on a Linux server and discover that you are getting errors about there being too many open files, then the standard advice is to set the ‘open_files_limit’ in my.cnf to a higher value. However, rarely is this option explained and it may not work as you expect.

The open_files_limit option does one thing. It sets the soft limit for open files (max files) but it can never be set higher than the hard limit, which is often imposed by the default Linux kernel configuration and set at an insanely low value. Unless the hard limit has been increased, setting the soft limit with open_files_limit in mysql may have little or no effect.

When you set the open_files_limit, either on the command line or in my.cnf, it tells mysqld to set the soft limit to the specified value – if it does not exceed the hard limit. Mysqld does this by calling the setrlimit() C function, which may or may not work depending on whether that function is available in your OS libraries. That is why it is best to use the mysqld_safe shell script to start mysqld, because that script will call ‘ulimit -n’ to set the limit in the shell before starting mysqld as extra insurance if you don’t happen to have setrlimit() on your system.

So, the question is not how to set the soft limit, but how to make sure your hard limit is high enough for the soft limit to work. To see the hard limit type:

# ulimit -Hn

4096

The soft limit you can see by typing:

# ulimit -Sn

1024

Those limits are typical of an Ubuntu defaults and not very good for a busy MySQL database installation. If you have a running instance of mysqld you can see what limits it is currently running with by typing:

# cat /proc/$(pgrep mysqld$)/limits

Limit                     Soft Limit           Hard Limit           Units     
Max cpu time              unlimited            unlimited            seconds   
Max file size             unlimited            unlimited            bytes     
Max data size             unlimited            unlimited            bytes     
Max stack size            8388608              unlimited            bytes     
Max core file size        0                    unlimited            bytes     
Max resident set          unlimited            unlimited            bytes     
Max processes             159663               159663               processes 
Max open files            196608               196608               files     
Max locked memory         65536                65536                bytes     
Max address space         unlimited            unlimited            bytes     
Max file locks            unlimited            unlimited            locks     
Max pending signals       159663               159663               signals   
Max msgqueue size         819200               819200               bytes     
Max nice priority         0                    0                    
Max realtime priority     0                    0                    
Max realtime timeout      unlimited            unlimited            us

Note: you can also run ‘ps aux’ to get a process list and use the pid (process id) number directly rather than using pgrep.

We are taking an interest in the row labeled, ‘Max open files’. We can see both the hard limit for the process owner user as well as the soft limit it is currently running. In the example above I have already increased my open file limits to an acceptably high level for my application. How do we change the hard limit?

If you are the root user then you can very easily change the setting for the shell by using the ulimit shell command and giving it a value:

# ulimit -Hn 196608

196608

However, this will only set it for the current shell session and you will want something more permanent. The best option is to modify your limits configuration, which in a typical Ubuntu installation is in /etc/security/limits.conf. Open that file with your editor of choice and add the following lines to the bottom of the file:

*                hard    nofile          199680
*                soft    nofile          1024

# End of file

That will set the hard open file limit (nofile) to a respectable 199680 files and the default soft limit to 1024 for all users – which is what the asterisk/wildcard means. That 1024 default is fine, since mysqld_safe or mysqld will set the soft limit based on the open_files_limit option.

You can also set the fs.file-max using sysctl command:

# sysctl fs.file-max 199680

fs.file-max = 199680

Though the value might be affected by some other settings such as settings directly in config files in /etc/sysctl.d/ or in /etc/sysctl.conf itself.

Ultimately, you will know that your open file limit issue is truly resolved when you can run:

# cat /proc/$(pgrep mysqld$)/limits

… and see that your mysql process is actually running the soft limit you specified with the open_file_limit option. Just remember that you CANNOT set your open_file_limit to be higher than the hard limit configured in the operation system.

How Ubuntu et al make Perl annoying with locale errors

Tags

Install, Locale, Perl, Ubuntu

Install a fresh instance of Ubuntu and you will most likely get that infuriating LC_ALL locale warnings whenever you use anything that invokes the Perl Interpreter. Why does it happen, and how do you fix it quickly?

It happens because a base install of Ubuntu, generated from disk image, which is the usual way cloud and VPS providers distribute OS packages, avoid making assumptions about my ethnicity and language of choice, that’s something they leave up to me to install myself. We don’t have nice little install wizards to select our language preferences. They make the assumption that I’m not a numbskull who just wants life to be relatively convenient and hassle-free – a rather poor assumption on their part! Fixing it is easy:

> sudo apt-get install language-pack-en-base
> sudo dpkg-reconfigure locales

And that’s all you have to do. No messing around trying to set environment variables or any of that nonsense. Just install the language pack of your choice and reconfigure the locales. If you are installing a French server language-pack-fr-base I would assume would work, etc.

To test everything is fixed just run “perl -v” and it should just print the version info with no warnings.