Back to Freecodecamp

Tokenize a string with escaping

curriculum/challenges/english/blocks/rosetta-code-challenges/594faaab4e2a8626833e9c3d.md

latest3.0 KB
Original Source

--description--

Write a function or program that can split a string at each non-escaped occurrence of a separator character.

It should accept three input parameters:

<ul> <li>The <strong>string</strong></li> <li>The <strong>separator character</strong></li> <li>The <strong>escape character</strong></li> </ul>

It should output a list of strings.

Rules for splitting:

<ul> <li>The fields that were separated by the separators, become the elements of the output list.</li> <li>Empty fields should be preserved, even at the start and end.</li> </ul>

Rules for escaping:

<ul> <li>"Escaped" means preceded by an occurrence of the escape character that is not already escaped itself.</li> <li>When the escape character precedes a character that has no special meaning, it still counts as an escape (but does not do anything special).</li> <li>Each occurrences of the escape character that was used to escape something, should not become part of the output.</li> </ul>

Demonstrate that your function satisfies the following test-case:

Given the string

<pre>one^|uno||three^^^^|four^^^|^cuatro|</pre>

and using | as a separator and ^ as escape character, your function should output the following array:

<pre> ['one|uno', '', 'three^^', 'four^|cuatro', ''] </pre>

--before-each--

js
const testStr1 = 'one^|uno||three^^^^|four^^^|^cuatro|';
const res1 = ['one|uno', '', 'three^^', 'four^|cuatro', ''];

const testStr2 = 'a@&bcd&ef&&@@hi';
const res2 = ['a&bcd', 'ef', '', '@hi'];

const testStr3 = 'hello^|world|how^are^you^|';
const res3 = ['hello|world', 'howareyou|'];

--hints--

tokenize should be a function.

js
assert(typeof tokenize === 'function');

tokenize should return an array.

js
assert(typeof tokenize('a', 'b', 'c') === 'object');

tokenize('one^|uno||three^^^^|four^^^|^cuatro|', '|', '^') should return ['one|uno', '', 'three^^', 'four^|cuatro', '']

js
assert.deepEqual(tokenize(testStr1, '|', '^'), res1);

tokenize('a@&bcd&ef&&@@hi', '&', '@') should return ['a&bcd', 'ef', '', '@hi']

js
assert.deepEqual(tokenize(testStr2, '&', '@'), res2);

tokenize('hello^|world|how^are^you^|', '|', '^') should return ['hello|world', 'howareyou|']

js
assert.deepEqual(tokenize(testStr3, '|', '^'), res3);

--seed--

--seed-contents--

js
function tokenize(str, sep, esc) {
  return true;
}

--solutions--

js
// tokenize :: String -> Character -> Character -> [String]
function tokenize(str, charDelim, charEsc) {
  const dctParse = str.split('')
    .reduce((a, x) => {
      const blnEsc = a.esc;
      const blnBreak = !blnEsc && x === charDelim;
      const blnEscChar = !blnEsc && x === charEsc;

      return {
        esc: blnEscChar,
        token: blnBreak ? '' : (
          a.token + (blnEscChar ? '' : x)
        ),
        list: a.list.concat(blnBreak ? a.token : [])
      };
    }, {
      esc: false,
      token: '',
      list: []
    });

  return dctParse.list.concat(
    dctParse.token
  );
}