Release version 1.7.0

From now on, releases will be cut from develop, and merged directly into master.

Each release will be a tag on the master branch (e.g. 1.7.0).
A "release/1.7.0" branch will eventually be created only if/when a hotfix will
be needed.

Author: muxator

CHANGELOG.md

@@ -1,3 +1,13 @@
+# 1.7.0
+* FIX: `getLineHTMLForExport()` no longer produces multiple copies of a line. **WARNING**: this could potentially break some plugins
+* FIX: authorship of bullet points no longer changes when a second author edits them
+* FIX: improved Firefox compatibility (non printable keys)
+* FIX: `getPadPlainText()` was not working
+* REQUIREMENTS: minimum required Node version is 6.9.0 LTS. The next release will require at least Node 8.9.0 LTS
+* SECURITY: updated MySQL, Elasticsearch and PostgreSQL drivers
+* SECURITY: started updating deprecated code and packages
+* DOCS: documented --credentials, --apikey, --sessionkey. Better detailed contributors guidelines. Added a section on securing the installation
+
 # 1.6.6
  * FIX: line numbers are aligned with text again (broken in 1.6.4)
  * FIX: text entered between connection loss and reconnection was not saved
@@ -490,7 +500,7 @@
  * Plugin-specific settings in settings.json (finally allowing for things like a google analytics plugin)
  * Serve admin dashboard at /admin (still very limited, though)
  * Modify your settings.json through the newly created UI at /admin/settings
- * Fix: Import <ol>'s  as <ol>'s and not as <ul>'s!
+ * Fix: Import `<ol>` as `<ol>` and not as `<ul>`!
  * Added solaris compatibility (bin/installDeps.sh was broken on solaris)
  * Fix a bug with IE9 and Password Protected Pads using HTTPS
 

CONTRIBUTING.md

@@ -1,6 +1,34 @@
 # Contributor Guidelines
 (Please talk to people on the mailing list before you change this page, see our section on [how to get in touch](https://github.com/ether/etherpad-lite#get-in-touch))
 
+## Pull requests
+
+* the commit series in the PR should be _linear_ (it **should not contain merge commits**). This is necessary because we want to be able to [bisect](https://en.wikipedia.org/wiki/Bisection_(software_engineering)) bugs easily. Rewrite history/perform a rebase if necessary
+* PRs should be issued against the **develop** branch: we never pull directly into **master**
+* PRs **should not have conflicts** with develop. If there are, please resolve them rebasing and force-pushing
+* when preparing your PR, please make sure that you have included the relevant **changes to the documentation** (preferably with usage examples)
+* contain meaningful and detailed **commit messages** in the form:
+  ```
+  submodule: description
+  
+  longer description of the change you have made, eventually mentioning the
+  number of the issue that is being fixed, in the form: Fixes #someIssueNumber
+  ```
+* if the PR is a **bug fix**:
+  * the first commit in the series must be a test that shows the failure
+  * subsequent commits will fix the bug and make the test pass
+  * the final commit message should include the text `Fixes: #xxx` to link it to its bug report
+* think about stability: code has to be backwards compatible as much as possible. Always **assume your code will be run with an older version of the DB/config file**
+* if you want to remove a feature, **deprecate it instead**:
+  * write an issue with your deprecation plan
+  * output a `WARN` in the log informing that the feature is going to be removed
+  * remove the feature in the next version
+* if you want to add a new feature, put it under a **feature flag**:
+  * once the new feature has reached a minimal level of stability, do a PR for it, so it can be integrated early
+  * expose a mechanism for enabling/disabling the feature
+  * the new feature should be **disabled** by default. With the feature disabled, the code path should be exactly the same as before your contribution. This is a __necessary condition__ for early integration
+* think of the PR not as something that __you wrote__, but as something that __someone else is going to read__. The commit series in the PR should tell a novice developer the story of your thoughts when developing it
+
 ## How to write a bug report
 
 * Please be polite, we all are humans and problems can occur.
@@ -25,12 +53,6 @@ If you send logfiles, please set the loglevel switch DEBUG in your settings.json
 
 The logfile location is defined in startup script or the log is directly shown in the commandline after you have started etherpad.
 
-
-## Important note for pull requests
-**Pull requests should be issued against the develop branch**.  We never pull directly into master.
-
-**Our goal is to iterate in small steps. Release often, release early. Evolution instead of a revolution**
-
 ## General goals of Etherpad
 To make sure everybody is going in the same direction:
 * easy to install for admins and easy to use for people
@@ -93,6 +115,8 @@ You can build the docs e.g. produce html, using `make docs`. At some point in th
 ## Testing
 Front-end tests are found in the `tests/frontend/` folder in the repository. Run them by pointing your browser to `<yourdomainhere>/tests/frontend`.
 
+Back-end tests can be run from the `src` directory, via `npm test`.
+
 ## Things you can help with
 Etherpad is much more than software.  So if you aren't a developer then worry not, there is still a LOT you can do!  A big part of what we do is community engagement.  You can help in the following ways
  * Triage bugs (applying labels) and confirming their existance

README.md

@@ -1,8 +1,3 @@
-### This project is looking for a new project lead.  If you wish to help steer Etherpad forward please email [email protected]
-
-[![Deps](https://david-dm.org/ether/etherpad-lite.svg?branch=develop)](https://david-dm.org/ether/etherpad-lite)
-[![NSP Status](https://nodesecurity.io/orgs/etherpad/projects/635f6185-35c6-4ed7-931a-0bc62758ece7/badge)](https://nodesecurity.io/orgs/etherpad/projects/635f6185-35c6-4ed7-931a-0bc62758ece7)
-
 # A really-real time collaborative word processor for the web
 ![Demo Etherpad Animated Jif](https://i.imgur.com/zYrGkg3.gif "Etherpad in action on PrimaryPad")
 
@@ -13,6 +8,9 @@ Etherpad is a really-real time collaborative editor scalable to thousands of sim
 
 # Installation
 
+## Requirements
+- `nodejs` >= **6.9.0** (preferred: `nodejs` >= **8.9**)
+
 ## Uber-Quick Ubuntu
 ```
 curl -sL https://deb.nodesource.com/setup_9.x | sudo -E bash -
@@ -26,25 +24,26 @@ You'll need gzip, git, curl, libssl develop libraries, python and gcc.
 - *For Fedora/CentOS*: `yum install gzip git curl python openssl-devel && yum groupinstall "Development Tools"`
 - *For FreeBSD*: `portinstall node, npm, curl, git (optional)`
 
-Additionally, you'll need [node.js](https://nodejs.org) installed, Ideally the latest stable version, we recommend installing/compiling nodejs from source (avoiding apt).
+Additionally, you'll need [node.js](https://nodejs.org) installed (minimum required Node version: **6.9.0**).
+Ideally, the latest stable version is preferred. Please note that the packages offered on some operating systems are outdated. In those cases, we recommend installing nodejs from official archives or compiling it from source (avoiding yum/apt).
 
 **As any user (we recommend creating a separate user called etherpad):**
 
-1. Move to a folder where you want to install Etherpad. Clone the git repository `git clone git://github.com/ether/etherpad-lite.git`
-2. Change into the new directory containing the cloned source code `cd etherpad-lite`
+1. Move to a folder where you want to install Etherpad. Clone the git repository: `git clone git://github.com/ether/etherpad-lite.git`
+2. Change into the new directory containing the cloned source code: `cd etherpad-lite`
 
-Now, run `bin/run.sh` and open <http://127.0.0.1:9001> in your browser. 
+Now, run `bin/run.sh` and open <http://127.0.0.1:9001> in your browser.
 
-Update to the latest version with `git pull origin`. The next start with bin/run.sh will update the dependencies.
+Update to the latest version with `git pull origin`. The next start with `bin/run.sh` will update the dependencies.
 
 [Next steps](#next-steps).
 
 ## Windows
 
-### Prebuilt windows package
+### Prebuilt Windows package
 This package works out of the box on any windows machine, but it's not very useful for developing purposes...
 
-1. [Download the latest windows package](http://etherpad.org/#download)
+1. [Download the latest Windows package](http://etherpad.org/#download)
 2. Extract the folder
 
 Now, run `start.bat` and open <http://localhost:9001> in your browser. You like it? [Next steps](#next-steps).
@@ -63,17 +62,26 @@ Update to the latest version with `git pull origin`, then run `bin\installOnWind
 
 If cloning to a subdirectory within another project, you may need to do the following:
 
-1. Start the server manually (e.g. `node/node_modules/ep_etherpad-lite/node/server.js]`)
+1. Start the server manually (e.g. `node/node_modules/ep_etherpad-lite/node/server.js`)
 2. Edit the db `filename` in `settings.json` to the relative directory with the file (e.g. `application/lib/etherpad-lite/var/dirty.db`)
 3. Add auto-generated files to the main project `.gitignore`
 
 # Next Steps
 
 ## Tweak the settings
-You can initially modify the settings in `settings.json`. (If you need to handle multiple settings files, you can pass the path to a settings file to `bin/run.sh` using the `-s|--settings` option. This allows you to run multiple Etherpad instances from the same installation.)  Once you have access to your /admin section settings can be modified through the web browser.
+You can modify the settings in `settings.json`.
+If you need to handle multiple settings files, you can pass the path to a settings file to `bin/run.sh` using the `-s|--settings` option: this allows you to run multiple Etherpad instances from the same installation.
+Similarly, `--credentials` can be used to give a settings override file, `--apikey` to give a different APIKEY.txt file and `--sessionkey` to give a non-default SESSIONKEY.txt.
+Once you have access to your /admin section settings can be modified through the web browser.
 
 You should use a dedicated database such as "mysql", if you are planning on using etherpad-in a production environment, since the "dirtyDB" database driver is only for testing and/or development purposes.
 
+## Secure your installation
+If you have enabled authentication in `users` section in `settings.json`, it is a good security practice to **store hashes instead of plain text passwords** in that file. This is _especially_ advised if you are running a production installation.
+
+Please install [ep_hash_auth plugin](https://www.npmjs.com/package/ep_hash_auth) and configure it.
+If you prefer, `ep_hash_auth` also gives you the option of storing the users in a custom directory in the file system, without having to edit `settings.json` and restart Etherpad each time.
+
 ## Plugins and themes
 
 Etherpad is very customizable through plugins. Instructions for installing themes and plugins can be found in [the plugin wiki article](https://github.com/ether/etherpad-lite/wiki/Available-Plugins).

bin/backendTests.sh

@@ -1,5 +1,10 @@
 #!/bin/bash
 #
+# WARNING: since Etherpad 1.7.0 (2018-08-17), this script is DEPRECATED, and
+#          will be removed/modified in a future version.
+#          It's left here just for documentation.
+#          The branching policies for releases have been changed.
+#
 # This script is used to publish a new release/version of etherpad on github
 #
 # Work that is done by this script:
@@ -16,6 +21,16 @@
 # ETHER_REPO:
 # - Create a new release on github
 
+printf "WARNING: since Etherpad 1.7.0 this script is DEPRECATED, and will be removed/modified in a future version.\n\n"
+while true; do
+    read -p "Do you want to continue? This is discouraged. [y/N]" yn
+    case $yn in
+        [Yy]* ) break;;
+        [Nn]* ) exit;;
+        * ) printf "Please answer yes or no.\n\n";;
+    esac
+done
+
 ETHER_REPO="https://github.com/ether/etherpad-lite.git"
 ETHER_WEB_REPO="https://github.com/ether/ether.github.com.git"
 TMP_DIR="/tmp/"

bin/createRelease.sh

@@ -1,5 +1,53 @@
 #!/bin/sh
 
+# minimum required node version
+REQUIRED_NODE_MAJOR=6
+REQUIRED_NODE_MINOR=9
+
+# minimum required npm version
+REQUIRED_NPM_MAJOR=3
+REQUIRED_NPM_MINOR=10
+
+require_minimal_version() {
+  PROGRAM_LABEL="$1"
+  VERSION_STRING="$2"
+  REQUIRED_MAJOR="$3"
+  REQUIRED_MINOR="$4"
+
+  # Flag -s (--only-delimited on GNU cut) ensures no string is returned
+  # when there is no match
+  DETECTED_MAJOR=$(echo $VERSION_STRING | cut -s -d "." -f 1)
+  DETECTED_MINOR=$(echo $VERSION_STRING | cut -s -d "." -f 2)
+
+  if [ -z "$DETECTED_MAJOR" ]; then
+    printf 'Cannot extract %s major version from version string "%s"\n' "$PROGRAM_LABEL" "$VERSION_STRING" >&2
+    exit 1
+  fi
+
+  if [ -z "$DETECTED_MINOR" ]; then
+    printf 'Cannot extract %s minor version from version string "%s"\n' "$PROGRAM_LABEL" "$VERSION_STRING" >&2
+    exit 1
+  fi
+
+  case "$DETECTED_MAJOR" in
+      ''|*[!0-9]*)
+        printf '%s major version from "%s" is not a number. Detected: "%s"\n' "$PROGRAM_LABEL" "$VERSION_STRING" "$DETECTED_MAJOR" >&2
+        exit 1
+        ;;
+  esac
+
+  case "$DETECTED_MINOR" in
+      ''|*[!0-9]*)
+        printf '%s minor version from "%s" is not a number. Detected: "%s"\n' "$PROGRAM_LABEL" "$VERSION_STRING" "$DETECTED_MINOR" >&2
+        exit 1
+  esac
+
+  if [ "$DETECTED_MAJOR" -lt "$REQUIRED_MAJOR" ] || ([ "$DETECTED_MAJOR" -eq "$REQUIRED_MAJOR" ] && [ "$DETECTED_MINOR" -lt "$REQUIRED_MINOR" ]); then
+    printf 'Your %s version "%s" is too old. %s %d.%d.x or higher is required.\n' "$PROGRAM_LABEL" "$VERSION_STRING" "$PROGRAM_LABEL" "$REQUIRED_MAJOR" "$REQUIRED_MINOR" >&2
+    exit 1
+  fi
+}
+
 #Move to the folder where ep-lite is installed
 cd `dirname $0`
 
@@ -36,22 +84,15 @@ hash npm > /dev/null 2>&1 || {
 }
 
 #Check npm version
-NPM_VERSION=$(npm --version)
-NPM_MAIN_VERSION=$(echo $NPM_VERSION | cut -d "." -f 1)
-if [ $(echo $NPM_MAIN_VERSION) = "0" ]; then
-  echo "You're running a wrong version of npm, you're using $NPM_VERSION, we need 1.x or higher" >&2
-  exit 1
-fi
+NPM_VERSION_STRING=$(npm --version)
+
+require_minimal_version "npm" "$NPM_VERSION_STRING" "$REQUIRED_NPM_MAJOR" "$REQUIRED_NPM_MINOR"
 
 #Check node version
-NODE_VERSION=$(node --version)
-NODE_V_MINOR=$(echo $NODE_VERSION | cut -d "." -f 1-2)
-NODE_V_MAIN=$(echo $NODE_VERSION | cut -d "." -f 1)
-NODE_V_MAIN=${NODE_V_MAIN#"v"}
-if [ ! $NODE_V_MINOR = "v0.10" ] && [ ! $NODE_V_MINOR = "v0.11" ] && [ ! $NODE_V_MINOR = "v0.12" ] && [ ! $NODE_V_MAIN -ge 4 ]; then
-  echo "You're running a wrong version of node. You're using $NODE_VERSION, we need node v0.10.x or higher" >&2
-  exit 1
-fi
+NODE_VERSION_STRING=$(node --version)
+NODE_VERSION_STRING=${NODE_VERSION_STRING#"v"}
+
+require_minimal_version "nodejs" "$NODE_VERSION_STRING" "$REQUIRED_NODE_MAJOR" "$REQUIRED_NODE_MINOR"
 
 #Get the name of the settings file
 settings="settings.json"
@@ -73,7 +114,7 @@ echo "Ensure that all dependencies are up to date...  If this is the first time
   cd node_modules
   [ -e ep_etherpad-lite ] || ln -s ../src ep_etherpad-lite
   cd ep_etherpad-lite
-  npm install --loglevel warn
+  npm install --no-save --loglevel warn
 ) || {
   rm -rf node_modules
   exit 1

bin/installDeps.sh

@@ -72,7 +72,7 @@ The API is accessible via HTTP. HTTP Requests are in the format /api/$APIVERSION
 ### Response Format
 Responses are valid JSON in the following format:
 
-```js
+```json
 {
   "code": number,
   "message": string,

doc/api/http_api.md

@@ -50,7 +50,7 @@ There are server hooks, which will be executed on the server (e.g. `expressCreat
 ### Parts
 As your plugins become more and more complex, you will find yourself in the need to manage dependencies between plugins. E.g. you want the hooks of a certain plugin to be executed before (or after) yours. You can also manage these dependencies in your plugin definition file `ep.json`:
 
-```javascript
+```json
 {
   "parts": [
     {
@@ -99,7 +99,7 @@ Your plugin must also contain a [package definition file](https://docs.npmjs.com
   "author": "USERNAME (REAL NAME) <[email protected]>",
   "contributors": [],
   "dependencies": {"MODULE": "0.3.20"},
-  "engines": { "node": ">= 0.6.0"}
+  "engines": { "node": ">= 6.9.0"}
 }
 ```
 

doc/plugins.md

@@ -15,4 +15,4 @@ We currently measure:
 
 Under the hood, we are happy to rely on [measured](https://github.com/felixge/node-measured) for all our metrics needs.
 
-To modify or simply access our stats in your plugin, simply `require('ep_etherpad-lite/stats')` which is a `measured.Collection`.
+To modify or simply access our stats in your plugin, simply `require('ep_etherpad-lite/stats')` which is a [`measured.Collection`](https://yaorg.github.io/node-measured/packages/measured-core/Collection.html).