Messaging continues to be of rise. The new generation is more willing to
send texts than to call. Communicating with an instant messenger has an
unique advantage over the old good email: you can easily send replies
over replies quickly, resulting in a dialogue. But there is a serious problem:
many of the instant messengers are commercial products that work such that
their "users" are in fact the exploitable resource having no control or
choice.
Most corporations are fair providers of various products and services we
can buy. But not these "Big Tech" that offer "free applications," including
instant messengers. There is, obviously, nothing free on the Earth. Then,
if you do not pay, then you are the product not the customer. The Big
Tech corporations exploit the "end-users" to suck out private data,
often for further resale. Nearly all of these messengers have centralised
architecture and the user's account is linked to the telephone number,
completely destroying privacy. The link to the telephone number is also
very inconvenient because you cannot get several accounts easily, this
requires obtaining several mobile subscriptions. It's just illogical,
expensive and silly. Centralized architecture dictates that the
communication is kept on the corporate servers
so theoretically many employees can read
messages by abuse.
Some of the products are advertised as end-to-end encrypted. But nearly
all of them are closed source so there is no way to check how this is
implemented and if and when the service owner can have access to private
messages content. Moreover, we have evidence for the opposite. Many
so called "end-to-end encrypted" messages are actually read by AI and human
contractors.
Even if communication is technically end-to-end encrypted, the company owns
and fully controls the server, the client application and network traffic, so
a man-in-the-middle attack by silently changing certificates is possible
(e.g. in the context of lawful intercept, or unlawful abuse). Metadata
(technical information information about all aspects of communication,
including the addressees, their locations, IP addresses, telephone number
etc.) is always accessible to the service. But metadata is often even
more informative than the message content. How such metadata is used is
typically unclear. The user has no authority here at all.
Nearly all of these messengering systems have closed proprietary protocol. This
means that how you use the product is completely controlled by the owner
company. The only way to use the product is with the official application.
You cannot just choose for yourself which application program to use. This
is cardinally different from the email, for example, where you can use the
provider's web interface, its mobile app or any of the many available email
applications such as Thunderbird or
K-9 Mail. With such a third-party application you
can easily consolidate several email accounts in one place and easily make
use of the functionality the provider does not offer, such as end-to-end
encryption. Another
major problem is monopoly and lack of interoperability. The "users" (in
reality, the exploited resource) are completely restricted to the owner's
platform and are unable to communicate with the other (especially competing)
platforms (e.g. Facebook to Snapchat) as a way to keep users within the silo.
This is as if you were unable to call/send sms across different mobile
operators. And this is silly. To break down monopoly, ensure fairer
competition and interoperability across the services, the EU has developed
the Digital Markets Act (DMA) regulation.
This is a big step, but it does not solve many of the problems with
centralization, privacy and regular security flaws.
Take back your freedom, privacy and security
So, why use the restricted, inconvenient, monopolistic, insecure and non-private platforms for
the trivial task of sending instant messages? There are several ways
to configure one's own privately controlled instant messaging system:
XMPP and Matrix. XMPP is
lightweight and more private, yet covers all the typical instant communication
purposes: text, file share and voice. Moreover, XMPP servers are by default
federated:
it is easy to send messages across the different servers like in the
email. There are many different applications for all operating systems and
platforms the user can choose.
It is very easy to set up one's own XMPP server for a small group,
company,
the family or just an individual. You will need
two things:
-
Server that will be the central hub for the communication network running
24x7. This can be anything, from a Rasberry PI in a cupboard to a Virtual
Private Server
(VPS) somewhere in a data centre or just an old PC running in your
basement. A small scale VPS useful for an XMPP server can be very cheap,
up to a three Euro per month. There exist even cheaper options, such as
EUR 6 per year. There are also dedicated search engines
to help locate cheap VPS, e.g. LowendBox
and ServerHunter.
A typical operating system running on the server is Linux (very secure,
highly configurable, free and open source).
-
Domain name that needs to be used to connect to the XMPP server. Domain
can be registered to the user (e.g. myname.no), which costs about
30 Euro yearly. But a sub-domain can be obtained for free using the
https://freedns.afraid.org or similar
"free DNS" services. In the later case you might have something like
myownchat.mooo.com or myownchat.ptchat.net. It is possible to run the
XMPP server purely on IP address even without domain name, but it is much
less convenient (e.g. then federation with other servers is lost).
Given you have got a server (VPS or dedicated machine) and the domain,
configuring an XMPP server can be done on 1-2-3. There exist several Linux
variants (distributives) with different management commands (usually for
installing software). I assume Debian Linux
is used below (the same commands also work for Ubuntu and other Debian-based
Linux systems).
1. Install XMPP server software
Login. When you have got a server of any kind, you need tologin
to it, typically with ssh:
here the user name on the server is debian and the server ip
is 1.2.3.4. Typically, you may need to create the ssh key and
upload it to the server to authenticate (refer the server documentation, e.g.
this).
I assume logging-in is not a problem.
Prepare server. First of all, update the software on the new server
sudo apt update -y && sudo apt-get upgrade -y
Install some useful monitoring and security-enhancing utilities
sudo apt install -y mc htop atop nload nmon tree zip pwgen fail2ban dnsutils iptables-persistent locate unattended-upgrades
Install certbot, a system that manages the
TLS certificates
for secure connection
sudo apt -y install certbot
Install the ejabberd server, which is is very
reliable and light on resources
sudo apt install ejabberd
Firewall. To allow incoming network access to this server by the XMPP
clients and also third-party servers, the server needs to configure
the firewall rules. This can be done differently in different
installations. For example, some VPS may do this using a friendly web
interface. The standard Linux firewall is done via iptables.
The XMPP system requires incoming acces via ports 5222, 5223, 5269, 5443,
5280, 3478. To determine the ports refer to the listen section of the XMPP
configuration file below.
sudo iptables -A INPUT -p tcp --dport 5222 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
sudo iptables -A INPUT -p tcp --dport 5223 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
sudo iptables -A INPUT -p tcp --dport 5269 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
sudo iptables -A INPUT -p tcp --dport 5443 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
sudo iptables -A INPUT -p tcp --dport 5280 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
# STUN is over udp
sudo iptables -A INPUT -p udp --dport 3478 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
The port 7777 is used for a proxy for peer-to-peer (bytestream) file
transfer. If peer-to-peer file sharing is intended for use, an additional
rule should be set allowing incoming connections:
sudo iptables -A INPUT -p tcp --dport 7777 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
To see what firewall rules are in effect issue this:
iptables -L --line-numbers
It makes sense to save the iptables rules so they are automatically get in
effect after reboot
iptables-save > /etc/iptables/rules.v4
2. Configure your XMPP server
Secure connection certificate. Get a free
Let's Encrypt
TLS certificate.
I assume you have got a free domain myownchat.ptchat.net from
https://freedns.afraid.org.
Note that ejabberd can manage (issue and update) TLS certificates on its
own, but this needs some configuration as described in the
acme configuration option:
https://docs.ejabberd.im/admin/configuration/basic/#acme.
An advantage of the standalone certificate management system (as here) is
that it is slightly less tricky and can easily be used with a
web server on the same machine.
Why not also configure a web server for a small static web site here?
Ejabberd is very lightweight and will happily coexist with many other
servers running on the same machine.
sudo certbot --standalone certonly -d myownchat.ptchat.net
This command will ask a few questions and issue a TLS certificate. This process
is done over http so http port 80 must allow incoming connections. If this is
not so, use the following command:
sudo iptables -A INPUT -p tcp --dport 80 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
Do not forget to save iptables rules with the iptables-save as above.
The certificate files are located in
/etc/letsencrypt/live/myownchat.ptchat.net/fullchain.pem directory.
For the sake of security, the certificate directories have by default no
access to anyone except the admin (root) user. But this precludes the XMPP
server ejabberd to access the certificate. This can be easily fixed with the
following commands
First, add ejabberd to the root group
sudo adduser ejabberd root
Second, allow access to the certificate directories to the group
sudo chmod g+rx /etc/letsencrypt/live/myownchat.ptchat.net
sudo chmod g+rx /etc/letsencrypt/live
sudo chmod g+rx /etc/letsencrypt/
Configure ejabberd. Once the preparations are done, it is time to
configure the ejabberd messaging server. Edit the configuration file
(assuming the mcedit text editor is used)
sudo mcedit /etc/ejabberd/ejabberd.yml
This is a long configuration file that may look scary. But in fact only a few
changes are required to make the server running with the default options. But
note that the indents are important, try to keep them as in the original file.
Any line starting with # is considered a comment, this can be easily used
to disable specific options by "commenting them out."
First, set up the host name that is used for the server, it is the same as
the domain:
hosts:
- myownchat.ptchat.net
Second, configure the location of the TLS certificates that are used by the
server:
certfiles:
- "/etc/letsencrypt/live/myownchat.ptchat.net/fullchain.pem"
- "/etc/letsencrypt/live/myownchat.ptchat.net/privkey.pem"
Configure the admin users who can manage the XMPP server:
acl:
admin:
user:
- ""
- "myname": "myownchat.ptchat.net"
Then, add configuration for http-file-upload module that will allow file
sharing (sending files):
mod_http_upload:
put_url: https://@HOST@:5443/upload
custom_headers:
"Access-Control-Allow-Origin": "https://@HOST@"
"Access-Control-Allow-Methods": "GET,HEAD,PUT,OPTIONS"
"Access-Control-Allow-Headers": "Content-Type"
It is convenient to keep the latest messages on the server, it is done with
the "mam" module:
mod_mam:
assume_mam_usage: true
default: always
Ejabberd supports several other communication protocols in addition to
XMPP. For example, it also works with MQTT that is
typically used for IoT devices. If this functionality is not used,
just comment out the MQTT module to disable it.
The STUN and TURN protocol is mainly used for voice calls and needs the
actual IP address of the server (replace with your server IP addfress)
-
port: 3478
ip: "::"
transport: udp
module: ejabberd_stun
use_turn: true
## The server's public IPv4 address:
turn_ipv4_address: "1.2.3.4"
An important issue is wether to allow anonymous registrations of new users.
I strongly recommend not allowing this for security reasons. For a small
private server, you will normally add users manually and set them initial
passwords. Every user can then change password within the client program. So,
you need to disable the mod_register by commenting it out:
# mod_register:
# ## Only accept registration requests from the "trusted"
# ## network (see access_rules section above).
# ## Think twice before enabling registration from any
# ## address. See the Jabber SPAM Manifesto for details:
# ## https://github.com/ge0rg/jabber-spam-fighting-manifesto
# ip_access: trusted_network
Start server! And that's all minimal configuration. Now it's time to
start the server:
sudo systemctl start ejabberd
If there are any errors and the server fails to start, Linux logs can be
inspected with this command:
or logs for only ejabberd:
sudo journalctl -xe --unit ejabberd
Additional stuff. The above is enough to make the XMPP server running for
text. If voice is required, you need to configure the DNS as described here:
https://www.process-one.net/blog/how-to-set-up-ejabberd-video-voice-calling/.
DNS is normally configured using the control panel of the domain registrar.
The TLS certificate that is managed by certbot
is updated each 90 days. This is an automatic process, but the ejabberd
server must know when certificate is changed. This can be done using the
deploy hook. Just create the hook file reloadxmpp.sh (the file name can be
anything):
sudo mcedit /etc/letsencrypt/renewal-hooks/deploy/reloadxmpp.sh
and add the following commands:
#!/bin/sh
ejabberdctl reload_config
This file must be executable, so issue this command:
sudo chmod ugo+x /etc/letsencrypt/renewal-hooks/deploy/reloadxmpp.sh
The last note on the server is that it should be regularly updated for
bug fixes and security updates. This is done automatically by installing
unattended-upgrades above. Yet, it is a good practice to log in regularly
over the ssh, check logs and update the system:
sudo apt update -y && sudo apt-get upgrade -y
3. Configure the XMPP users and client application
Register new users. First, you need to register the XMPP users. The
quickest method is to use the command line on the server, the command
ejabberdctl has advanced functions.
A secure random password can be generated withy pwgen, e.g. the following
generates passwords with 18 symbols:
It normally generates an array of possible passwords to choose from.
Now, to register the user myname, It is the admin user configured in the main
configuration file /etc/ejabberd/ejabberd.yml above.
# user domain password
sudo ejabberdctl register myname myownchat.ptchat.net pee8chogh9Heel6hei
Other users can be configured similarly. Note that the full user name for XMPP
has the same format se email: myname@myownchat.ptchat.net. This is due to
the federated nature of both systems: you need to know both the user and
the server with whom to communicate.
For this example let's register two additional users:
sudo ejabberdctl register john.dow myownchat.ptchat.net ohyeeLeefo9yief4gu
sudo ejabberdctl register anna.karenina myownchat.ptchat.net hejo7phiy2iFeW9She
Use! The final step is configure the client program on the
user's device. The biggest difficulty at this step is the plenty
of choice. For any major platform, one can choose any of the many
available XMPP client programs. Some email
programs, e.g. Thunderbird also support
XMPP (although only a limited subset of features). Check out the
https://xmpp.org. The configuration for the client
is simple:
-
Server: your server, in the example above it is myownchat.ptchat.net
-
User name: your user name. In the example we used above, it can be
myname
Note that the option to create new account must NOT be enabled as
long as the account has already been created on the sever and the in-band
registration (mod_register, see above) is disabled for
security.
Some programs accept the full user name without specifying user and domain
separately. Then the user is just myname@myownchat.ptchat.net. If you
plan to use the peer-to-peer (bytestream) file transfer (but
this is not mandatory), you should also find where the file transfer proxy is
configured and set it with the proxy subdomain, for our example it should be
proxy.myownchat.ptchat.net. And that is all for basic client configuration.
I recommend the Blabber XMPP application for
devices running Android. Yaxim is the best option for
minimalists, it is notoriously miniature (only a few megabytes) and works great
even on the oldest and weakest devices. Miranda NG
is a powerful XMPP client program for Windows. There are also a few
web-based clients: https://conversejs.org/ and
https://web.xabber.com/ that you can try right
away without installing anything.
The final step is to fill the contact list (called roster) with the addresses
of the people (or maybe devices, because XMPP can be easily configured for
bots accepting commands). Just remember that the address is full name as in
email: user@server.domain. One useful option is so called Shared roster
groups: then you can configure
a group of contacts without the need to add them manually.
Happy chatting!
Further
There are many advanced options and possibilities in ejabberd. Just check
the documentation at the official web site: https://www.ejabberd.im/
and documentation https://docs.ejabberd.im/.
There are also a few useful tutorials, e.g.
https://www.process-one.net/blog/how-to-move-the-office-to-real-time-im-on-ejabberd/
The old good email remains the most critical digital communication tool.
What makes the venerable email so useful and sustainable
over the long time is its openness and standardization. Email is radically
different from the modern "apps" which integrate all pieces of technology--the
server, the client, and the protocol--by a single monopolist provider. With
email, we are free to choose the server (provider) and client with any
combination. It provides enormous flexibility, added privacy and
security. Indeed, the provider does not control my client and cannot add
backdoors; there is no monoculture of client software with all the related
security risks (any security vulnerability is global). Email is one of
the few pieces of technology that is very resistant against internet
censorship. Repressive state can easily block a web site and even force
an app store to remove an app
(as the Navalny's "Smart Voting").
Also, an app store can delete it for any other bizarre reason. But it
is much more difficult to block a mailing list: it is easy to redeploy and
recreate it on a different server (without the users even noticing anything).
Furthermore, The user can easily create several different email-based
identities (e.g. a separate one for politically sensitive activity) which
adds anonymity. And anonymity means physical security in some countries.
It is not surprising that many internet services use the email address
to register users, authenticate, restore password and other similar
purposes. Open, standardized and decentralized email is one of the most
critical technology everything else depends on. After all, the flexibility
offered by the email technology--the freedom to choose all pieces (provider,
client etc.) is just very very handy, at least for an advanced user (you
can add new features on top of what the provider realized, even against the
provider's will--isn't it convenient?).
The whole email technology is build around open protocols rather than a
centralized platform. This facilitates competition, makes for better and
fairer service and reduce possible impacts of malicious monopolists
(Masnick, 2019).
Google's Gmail has long been one of the main pillars of email, millions used
to rely upon every day. We should praise Google for popularising email
as the basic mainstream technology among the masses. I started using Gmail
many years ago when it was in its "beta" and available only by invitation.
At that time Gmail openness and unrestricted nature was just blazing. The
web interface was lightweight and not really cluttered with ugly banners,
unlike other email providers. There were ads but they were small and
unobtrusive. Gmail had long supported all the basic protocols (POP, IMAP,
SMTP) that allowed to use any standard compliant client software, and that
was available for free (some other providers were more greedy and allowed
this only on paid plans). Google's POP, IMAP and SMTP implementations
have been (and still remain!) quite idiosyncratic, incomplete and not
really standard-compliant which caused various glitches (e.g. message
deletion and default sorting are weird, I always hated Gmail's labels). But
this was bearable.
The serious privacy problems and threats of Gmail,
such as user email scanning for context-specific advertising
(until 2017)
or
AI tool
which could provide access to some pieces of data to third-party
developers.
That is nearly a disaster that cannot be fixed because spying on the user's
data is at the heart of Google's business model. But who cares as long as
it is free! I have long been using and promoting PGP encryption which
could fix many of the privacy (and security) problems.
Yes, PGP is crucial for individuals and businesses
and yes, a motivated user can encrypt.
Gmail still remained free and relatively open while an alternative of
deploying private email server is time-consuming and tedious (e.g. ensuring
that emails from a tiny private server don't end up in spam folders of
intended recipients). I used to pay with some of my privacy to get the
usability and stability of Gmail.
But over time I became increasingly concerned about the clear trend taken
by Google to make the open email more and more difficult to use outside of
the Google monopolistic ecosystem. There are signs of the famous
embrace, extend, and extinguish
strategy. Gmail API is featureful and powerful... but only if
you really need the complexity and like to play with the Google rules. If
you don't like to see ads, for example, and for this use a standard IMAP
mail client of your choice, your must suffer. If you need full PGP support on
a mobile client, never offered by Google, you are out of luck and have to
use an IMAP-based mobile app like Android K-9 Mail
that requires sacrificing some usability.
Google tends to draw its users by all means into its browser, its
own apps and APIs to get more user's private data and show ads. For
that matter, Google's security usability has become just terrible. The
intrusive access-blocks when a mobile user with an IMAP client moves across
IP addresses can drive anyone crazy... Access can be blocked even if the
user switches just to the next IP address within the same provider's IP
pool.
I have to use VPN with fixed IP address to avoid these stupid blocks!
To help keep your account secure, Google will no longer support the use
of third-party apps or devices which ask you to sign in to your Google
Account using only your username and password. Instead, you’ll need
to sign in using Sign in with Google.
The Google's insistence on rather complicated and heavyweight
OAuth2
mechanism
for basic email client access (remember, most email programs do not require
you to enter your password every time, diminishing the risk of phishing)
is understandable only as a means to limit all uncontrollable third-party
clients. Yes, OAuth2 is logical for complex workflows of data access delegation
across multiple web-based services with different login/password combinations
(the "Auth" stands for authorization, not
authentication).
Whenever I need access to my own emails I need to authenticate
my identity granting full access. But isn't OAuth2 client secret kept
on the device just as the username/password combination? Yet, limiting the
(power) users access to their own data provides just an illusion of
security at a large cost to usability and compatibility.
The Google's move to OAuth2 authorization seem to point that
the Gmail-hosted emails do not belong to me any more. My emails are now
owned by Google, who just "authorizes" (delegates) me access to some of
the data without trusting me. This is not what I need from my private
communication. Does Google pretend to "zero-trust" any third-party
apps? Maybe it doesn't trust its users (the owners of their data),
assuming they are all idiots?
If you think your users are idiots, only idiots will use it [your service]. ---
Linus Torvalds
And there is another side effect: as Google increasingly deployed more and
more heavyweight frameworks and technologies, Gmail became very sluggish
and bloated. It is cluttered and confusing, especially to those who don't
use it often enough to remember all the idiosyncrasies. And it's still poorly
adaptable to the user's needs. How can I get a fixed-width font for my plain
text message? Where is my favourite basic (and very fast) HTML web interface?
Enough is enough. I now go away from Gmail, and primarily not because of
big privacy concerns (which is quite expectable) but because of deteriorating
usability and growing incompatibility. It looks like the people at Google have
forgotten their old motto "Don't be evil." While I have been paying Google
with my privacy currency in the past to get functionality and usability,
the benefits of Gmail continuously went lower and now reached an unprofitable
level.
Migadu is my choice
There are many hosted email providers, some are focused on privacy and
security. For example, Protonmail is a fantastic
project that makes it nearly trivial to use PGP even for an uninitiated. But
its drawbacks are that it is non-standard and has too high publicity making
it quite undesirable in certain authoritarian countries. Simply said, if
you use Protonmail in some countries you may be suspected; Protonmail can
be blocked by the authorities, and worse still,
blocked in quite idiosyncratic way.
Some services may also reject registration
using this service.
What I have finally chosen is Migadu. It is not
yet another standard email hosting provider. It is a domain-based service. Once
you have got your own domain name (domains are now cheap), you can make your
own email service for your domain. That simple. This makes it super useful
for companies, families, groups and NGOs without large budgets. For a
reasonable price you get nearly your own mail server with many configurable
features (any custom mailboxes, aliases, forwarding, regexp, webmail,
etc.) but without the need to maintain all this complex system.
If you have a web site, you necessarily get a domain name for it. Now it's
easy to get your own email identity. True that some hosting providers also
do host email. But if you decide to switch to a different hosting it will
create a trouble: you need to move also email and this fact strongly limits
your next choice. Having a completely indpendent email system for your
existing domain avoids such hoster lock-in and makes life much easier.
By the way, the Migadu standard webmail interface
is sleek and very simple. Looks modern but lightweight and quite fast. No
bloat whatsoever, only the most crucial functionality. I am not big fan
of web-based email, but use it from time to time. And there is even some
very basic support for PGP! (But remember that web-based PGP is
not a very secure solution.)
I found the mail server configuration (including more esoteric stuff like
DNS setup and DKIM signatures) very easy. In my view you do not need an
IT degree to configure your email server with full functionality. I like the
admin panel, it is minimalist and easy to use, no stupid and distracting
visual effects. And Migadu is advertised as
fully open standard compliant service without proprietary glitches and
limitations. So any standard (open source or closed source) software is very
likely to be fully usable. This freedom is very important. And they are also
clear and honest about the
limitations and drawbacks.
Finally, goodbye Gmail.
PS: Disclaimer: I have no links with Migadu.
This post is also published on
Substack
and Medium
