Commit Graph

338 Commits

Author SHA1 Message Date
Leon
65d0b5b9d9
Make get hole errors and cell path hole errors identical (improvement on #7002) (#7647)
# Description

This closes #7498, as well as fixes an issue reported in
https://github.com/nushell/nushell/pull/7002#issuecomment-1368340773

BEFORE:
```
〉[{foo: 'bar'} {}] | get foo
Error: nu:🐚:column_not_found (link)

  × Cannot find column
   ╭─[entry #5:1:1]
 1 │ [{foo: 'bar'} {}] | get foo
   · ────────┬────────   ─┬─
   ·         │            ╰── value originates here
   ·         ╰── cannot find column 'Empty cell'
   ╰────

〉[{foo: 'bar'} {}].foo
╭───┬─────╮
│ 0 │ bar │
│ 1 │     │
╰───┴─────╯
```
AFTER:
```
〉[{foo: 'bar'} {}] | get foo
Error: nu:🐚:column_not_found (link)

  × Cannot find column
   ╭─[entry #1:1:1]
 1 │ [{foo: 'bar'} {}] | get foo
   ·               ─┬        ─┬─
   ·                │         ╰── cannot find column 'foo'
   ·                ╰── value originates here
   ╰────

〉[{foo: 'bar'} {}].foo
Error: nu:🐚:column_not_found (link)

  × Cannot find column
   ╭─[entry #3:1:1]
 1 │ [{foo: 'bar'} {}].foo
   ·               ─┬  ─┬─
   ·                │   ╰── cannot find column 'foo'
   ·                ╰── value originates here       
   ╰────
```

EDIT: This also changes the semantics of `get`/`select` `-i` somewhat.
I've decided to leave it like this because it works more intuitively
with `default` and `compact`.
BEFORE:
```
〉[{a:1} {b:2} {a:3}] | select -i foo | to nuon
null
```
AFTER:
```
〉[{a:1} {b:2} {a:3}] | select -i foo | to nuon
[[foo]; [null], [null], [null]]
```

# User-Facing Changes

See above. EDIT: the issue with holes in cases like ` [{foo: 'bar'}
{}].foo.0` versus ` [{foo: 'bar'} {}].0.foo` has been resolved.

# Tests + Formatting

Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect` to check that you're using the standard code
style
- `cargo test --workspace` to check that all tests pass

# After Submitting

If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
2023-01-02 14:45:43 -08:00
Kian-Meng Ang
79000aa5e0
Fix typos by codespell (#7600)
# Description

Found via `codespell -S target -L
crate,ser,numer,falsy,ro,te,nd,bu,ndoes,statics,ons,fo,rouge,pard`

# User-Facing Changes

None.

# Tests + Formatting

None and done.

# After Submitting

None.
2022-12-26 02:31:26 -05:00
Reilly Wood
a43e66ef92
Add LRU regex cache (#7587)
Closes #7572 by adding a cache for compiled regexes of type
`Arc<Mutex<LruCache<String, Regex>>>` to `EngineState` .

The cache is limited to 100 entries (limit chosen arbitrarily) and
evicts least-recently-used items first.

This PR makes a noticeable difference when using regexes for
`color_config`, e.g.:
```bash
#first set string formatting in config.nu like:
string: { if $in =~ '^#\w{6}$' { $in } else { 'white' } }`

# then try displaying and exploring a table with many strings
# this is instant after the PR, but takes hundreds of milliseconds before
['#ff0033', '#0025ee', '#0087aa', 'string', '#4101ff', '#ff0033', '#0025ee', '#0087aa', 'string', '#6103ff', '#ff0033', '#0025ee', '#0087aa', 'string', '#6103ff', '#ff0033', '#0025ee', '#0087aa', 'string', '#6103ff', '#ff0033', '#0025ee', '#0087aa', 'string', '#6103ff','#ff0033', '#0025ee', '#0087aa', 'string', '#6103ff','#ff0033', '#0025ee', '#0087aa', 'string', '#6103ff','#ff0033', '#0025ee', '#0087aa', 'string', '#6103ff','#ff0033', '#0025ee', '#0087aa', 'string', '#6103ff','#ff0033', '#0025ee', '#0087aa', 'string', '#6103ff']
```

## New dependency (`lru`)
This uses [the popular `lru` crate](https://lib.rs/crates/lru). The new
dependency adds 19.8KB to a Linux release build of Nushell. I think this
is OK, especially since the crate can be useful elsewhere in Nu.
2022-12-23 14:30:04 -08:00
Leon
dd7b7311b3
Standardise the use of ShellError::UnsupportedInput and ShellError::TypeMismatch and add spans to every instance of the former (#7217)
# Description

* I was dismayed to discover recently that UnsupportedInput and
TypeMismatch are used *extremely* inconsistently across the codebase.
UnsupportedInput is sometimes used for input type-checks (as per the
name!!), but *also* used for argument type-checks. TypeMismatch is also
used for both.
I thus devised the following standard: input type-checking *only* uses
UnsupportedInput, and argument type-checking *only* uses TypeMismatch.
Moreover, to differentiate them, UnsupportedInput now has *two* error
arrows (spans), one pointing at the command and the other at the input
origin, while TypeMismatch only has the one (because the command should
always be nearby)
* In order to apply that standard, a very large number of
UnsupportedInput uses were changed so that the input's span could be
retrieved and delivered to it.
* Additionally, I noticed many places where **errors are not propagated
correctly**: there are lots of `match` sites which take a Value::Error,
then throw it away and replace it with a new Value::Error with
less/misleading information (such as reporting the error as an
"incorrect type"). I believe that the earliest errors are the most
important, and should always be propagated where possible.
* Also, to standardise one broad subset of UnsupportedInput error
messages, who all used slightly different wordings of "expected
`<type>`, got `<type>`", I created OnlySupportsThisInputType as a
variant of it.
* Finally, a bunch of error sites that had "repeated spans" - i.e. where
an error expected two spans, but `call.head` was given for both - were
fixed to use different spans.

# Example
BEFORE
```
〉20b | str starts-with 'a'
Error: nu:🐚:unsupported_input (link)

  × Unsupported input
   ╭─[entry #31:1:1]
 1 │ 20b | str starts-with 'a'
   ·   ┬
   ·   ╰── Input's type is filesize. This command only works with strings.
   ╰────

〉'a' | math cos
Error: nu:🐚:unsupported_input (link)

  × Unsupported input
   ╭─[entry #33:1:1]
 1 │ 'a' | math cos
   · ─┬─
   ·  ╰── Only numerical values are supported, input type: String
   ╰────

〉0x[12] | encode utf8
Error: nu:🐚:unsupported_input (link)

  × Unsupported input
   ╭─[entry #38:1:1]
 1 │ 0x[12] | encode utf8
   ·          ───┬──
   ·             ╰── non-string input
   ╰────
```
AFTER
```
〉20b | str starts-with 'a'
Error: nu:🐚:pipeline_mismatch (link)

  × Pipeline mismatch.
   ╭─[entry #1:1:1]
 1 │ 20b | str starts-with 'a'
   ·   ┬   ───────┬───────
   ·   │          ╰── only string input data is supported
   ·   ╰── input type: filesize
   ╰────

〉'a' | math cos
Error: nu:🐚:pipeline_mismatch (link)

  × Pipeline mismatch.
   ╭─[entry #2:1:1]
 1 │ 'a' | math cos
   · ─┬─   ────┬───
   ·  │        ╰── only numeric input data is supported
   ·  ╰── input type: string
   ╰────

〉0x[12] | encode utf8
Error: nu:🐚:pipeline_mismatch (link)

  × Pipeline mismatch.
   ╭─[entry #3:1:1]
 1 │ 0x[12] | encode utf8
   · ───┬──   ───┬──
   ·    │        ╰── only string input data is supported
   ·    ╰── input type: binary
   ╰────
```

# User-Facing Changes

Various error messages suddenly make more sense (i.e. have two arrows
instead of one).

# Tests + Formatting

Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect` to check that you're using the standard code
style
- `cargo test --workspace` to check that all tests pass

# After Submitting

If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
2022-12-23 01:48:53 -05:00
Leon
5b616770df
Make config.filesize_format/config.filesize_metric conflict resolution consistent (#7410)
# Description

Currently, `filesize_format`/`filesize_metric` conflicts are resolved as
follows: if the `filesize_format` ends in "ib", then that overrides
`filesize_metric`, otherwise, `filesize_metric` overrides
`filesize_format`. This removes this difficult-to-predict asymmetric
behaviour, and makes it so that `filesize_metric` always overrides
`filesize_format`.

This also adds tests for `$env.config.filesize.format` and
`$env.config.filesize.metric` values.

REMINDER: `filesize_metric` means "increments of 1000", and refers to
KB-MB-GB-TB etc.

# User-Facing Changes

See above.

# Tests + Formatting

Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect` to check that you're using the standard code
style
- `cargo test --workspace` to check that all tests pass

# After Submitting

If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.

Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
2022-12-22 08:46:55 -06:00
Reilly Wood
705f12c1d9
Fix cell path when getting columns of non-records (#7508)
A follow-up to #7497. That change made it so that `get foo` would
eliminate non-record rows; I think that was an unintentional and
undesirable side-effect.

Before #7497:
```bash
〉[$nothing, { item: "foo" }] | get item
╭───┬─────╮
│ 0 │     │
│ 1 │ foo │
╰───┴─────╯
```
After #7497:
```bash
〉[$nothing, {item: "foo"}] | get item
╭───┬─────╮
│ 0 │ foo │
╰───┴─────╯
```

After this PR:
```bash
〉[$nothing, { item: "foo" }] | get item
╭───┬─────╮
│ 0 │     │
│ 1 │ foo │
╰───┴─────╯
```

cc: @merelymyself
2022-12-17 09:14:12 -08:00
pwygab
2d07c6eedb
ensure get doesn't delve too deep in nested lists (#7497)
# Description

Fixes #7494.

```
/home/gabriel/CodingProjects/nushell〉[[{foo: bar}]] | get foo          12/16/2022 12:31:17 PM
Error: nu::parser::not_found (link)

  × Not found.
   ╭─[entry #1:1:1]
 1 │ [[{foo: bar}]] | get foo
   · ───────┬──────
   ·        ╰── did not find anything under this name
   ╰────

```

# User-Facing Changes

cell paths no longer drill into nested tables.

# Tests + Formatting

Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect` to check that you're using the standard code
style
- `cargo test --workspace` to check that all tests pass

# After Submitting

If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
2022-12-15 22:03:38 -08:00
Reilly Wood
e215fbbd08
Add helper method to check whether ctrl+c was pressed, adopt it (#7482)
I've been working on streaming and pipeline interruption lately. It was
bothering me that checking ctrl+c (something we want to do often) always
requires a bunch of boilerplate like:
```rust
use std::sync::atomic::Ordering;

if let Some(ctrlc) = &engine_state.ctrlc {
     if ctrlc.load(Ordering::SeqCst) {
          ...
```
I added a helper method to cut that down to:

```rust
if nu_utils::ctrl_c::was_pressed(&engine_state.ctrlc) {
    ...
```
2022-12-15 09:39:24 -08:00
Leon
ce78817f41
$env.config now always holds a record with only valid values (#7309)
# Description
Closes #7059. Rather than generate a new Record each time $env.config is
accessed (as described in that issue), instead `$env.config = ` now A)
parses the input record, then B) un-parses it into a clean Record with
only the valid values, and stores that as an env-var. The reasoning for
this is that I believe `config_to_nu_record()` (the method that performs
step B) will be useful in later PRs. (See below)

As a result, this also "fixes" the following "bug":
```
〉$env.config = 'butts'
$env.config is not a record
〉$env.config
butts
```
~~Instead, `$env.config = 'butts'` now turns `$env.config` into the
default (not the default config.nu, but `Config::default()`, which
notably has empty keybindings, color_config, menus and hooks vecs).~~

This doesn't attempt to fix #7110. cc @Kangaxx-0

# Example of new behaviour

OLD:
```
〉$env.config = ($env.config | merge { foo: 1 })
$env.config.foo is an unknown config setting
〉$env.config.foo
1
```
NEW:
```
〉$env.config = ($env.config | merge { foo: 1 })
Error:
  × Config record contains invalid values or unknown settings

Error:
  × Error while applying config changes
   ╭─[entry #1:1:1]
 1 │ $env.config = ($env.config | merge { foo: 1 })
   ·                                           ┬
   ·                                           ╰── $env.config.foo is an unknown config setting
   ╰────
  help: This value has been removed from your $env.config record.

〉$env.config.foo
Error: nu:🐚:column_not_found (link)

  × Cannot find column
   ╭─[entry #1:1:1]
 1 │ $env.config = ($env.config | merge { foo: 1 })
   ·                              ──┬──
   ·                                ╰── value originates here
   ╰────
   ╭─[entry #2:1:1]
 1 │ $env.config.foo
   ·             ─┬─
   ·              ╰── cannot find column 'foo'
   ╰────
```
# Example of new errors

OLD:
```
$env.config.cd.baz is an unknown config setting
$env.config.foo is an unknown config setting
$env.config.bar is an unknown config setting
$env.config.table.qux is an unknown config setting
$env.config.history.qux is an unknown config setting
```
NEW:
```
Error: 
  × Config record contains invalid values or unknown settings

Error:
  × Error while applying config changes
     ╭─[C:\Users\Leon\AppData\Roaming\nushell\config.nu:267:1]
 267 │     abbreviations: true # allows `cd s/o/f` to expand to `cd some/other/folder`
 268 │     baz: 3,
     ·          ┬
     ·          ╰── $env.config.cd.baz is an unknown config setting
 269 │   }
     ╰────
  help: This value has been removed from your $env.config record.

Error:
  × Error while applying config changes
     ╭─[C:\Users\Leon\AppData\Roaming\nushell\config.nu:269:1]
 269 │   }
 270 │   foo: 1,
     ·        ┬
     ·        ╰── $env.config.foo is an unknown config setting
 271 │   bar: 2,
     ╰────
  help: This value has been removed from your $env.config record.

Error:
  × Error while applying config changes
     ╭─[C:\Users\Leon\AppData\Roaming\nushell\config.nu:270:1]
 270 │   foo: 1,
 271 │   bar: 2,
     ·        ┬
     ·        ╰── $env.config.bar is an unknown config setting
     ╰────
  help: This value has been removed from your $env.config record.

Error:
  × Error while applying config changes
     ╭─[C:\Users\Leon\AppData\Roaming\nushell\config.nu:279:1]
 279 │     }
 280 │     qux: 4,
     ·          ┬
     ·          ╰── $env.config.table.qux is an unknown config setting
 281 │   }
     ╰────
  help: This value has been removed from your $env.config record.

Error:
  × Error while applying config changes
     ╭─[C:\Users\Leon\AppData\Roaming\nushell\config.nu:285:1]
 285 │     file_format: "plaintext" # "sqlite" or "plaintext"
 286 │  qux: 2
     ·       ┬
     ·       ╰── $env.config.history.qux is an unknown config setting
 287 │   }
     ╰────
  help: This value has been removed from your $env.config record.


```

# User-Facing Changes

See above.

# Tests + Formatting

Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect` to check that you're using the standard code
style
- `cargo test --workspace` to check that all tests pass

# After Submitting

If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
2022-12-10 15:34:46 +02:00
Leon
220b105efb
Reduced LOC by replacing several instances of Value::Int {}, Value::Float{}, Value::Bool {}, and Value::String {} with Value::int(), Value::float(), Value::boolean() and Value::string() (#7412)
# Description

While perusing Value.rs, I noticed the `Value::int()`, `Value::float()`,
`Value::boolean()` and `Value::string()` constructors, which seem
designed to make it easier to construct various Values, but which aren't
used often at all in the codebase. So, using a few find-replaces
regexes, I increased their usage. This reduces overall LOC because
structures like this:
```
Value::Int {
  val: a,
  span: head
}
```
are changed into
```
Value::int(a, head)
```
and are respected as such by the project's formatter.
There are little readability concerns because the second argument to all
of these is `span`, and it's almost always extremely obvious which is
the span at every callsite.

# User-Facing Changes

None.

# Tests + Formatting

Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect` to check that you're using the standard code
style
- `cargo test --workspace` to check that all tests pass

# After Submitting

If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
2022-12-09 11:37:51 -05:00
Leon
9b41f9ecb8
Allow $env and mutable records to be mutated by = (closes #7110) (#7318)
# Description

Closes #7110. ~~Note that unlike "real" `mut` vars, $env can be deeply
mutated via stuff like `$env.PYTHON_IO_ENCODING = utf8` or
`$env.config.history.max_size = 2000`. So, it's a slightly awkward
special case, arguably justifiable because of what $env represents (the
environment variables of your system, which is essentially "outside"
normal Nushell regulations).~~
EDIT: Now allows all `mut` vars to be deeply mutated using `=`, on
request.

# User-Facing Changes

See above.

# Tests + Formatting

Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect` to check that you're using the standard code
style
- `cargo test --workspace` to check that all tests pass

# After Submitting

If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.
2022-12-06 19:51:55 +02:00
Daniel Buch Hansen
850ecf648a
Protocol: debug_assert!() Span to reflect a valid slice (#6806)
Also enforce this by #[non_exhaustive] span such that going forward we
cannot, in debug builds (1), construct invalid spans.

The motivation for this stems from #6431 where I've seen crashes due to
invalid slice indexing.

My hope is this will mitigate such senarios

1. https://github.com/nushell/nushell/pull/6431#issuecomment-1278147241

# Description

(description of your pull request here)

# Tests

Make sure you've done the following:

- [ ] Add tests that cover your changes, either in the command examples,
the crate/tests folder, or in the /tests folder.
- [ ] Try to think about corner cases and various ways how your changes
could break. Cover them with tests.
- [ ] If adding tests is not possible, please document in the PR body a
minimal example with steps on how to reproduce so one can verify your
change works.

Make sure you've run and fixed any issues with these commands:

- [x] `cargo fmt --all -- --check` to check standard code formatting
(`cargo fmt --all` applies these changes)
- [ ] `cargo clippy --workspace --features=extra -- -D warnings -D
clippy::unwrap_used -A clippy::needless_collect` to check that you're
using the standard code style
- [ ] `cargo test --workspace --features=extra` to check that all the
tests pass

# Documentation

- [ ] If your PR touches a user-facing nushell feature then make sure
that there is an entry in the documentation
(https://github.com/nushell/nushell.github.io) for the feature, and
update it if necessary.
2022-12-03 11:44:12 +02:00
Reilly Wood
3ac36879e0
Handle ctrl-c in RawStream iterator (#7314)
Fixes #7246 and #1898.

Darren noticed that `open /dev/random` could not be interrupted by
`ctrl+c`. Thankfully the solution was very simple; it looks like we just
forgot to check `ctrlc` in the `impl Iterator for RawStream`!

To reproduce this, just run `open /dev/random` and then cancel it with
`ctrl+c`.
2022-12-02 08:00:56 -08:00
Stefan Holderbach
fa97e819eb
Suggest using float on Value::Int overflow (#7253)
# Description

Suggest that floats support a larger range of values but warn about the
loss in precision.


![image](https://user-images.githubusercontent.com/15833959/204114339-c987cd47-f035-4c01-853f-e9a00441bf49.png)


(Doesn't apply to the types with associated units)

# Tests + Formatting

(-)
2022-11-29 13:30:02 -08:00
Stefan Holderbach
2ccb91dc6a
Add logical xor operator (#7242)
We already have the binary `bit-xor` and the shortcircuiting logical
`or`(`||`) and `and`(`&&`).
This introduces `xor` as a compact form for both brevity and clarity.
You can express the operation through `not`/`and`/`or` with a slight
risk of introducing bugs through typos.

Operator precedence

`and` > `xor` > `or`

Added logic and precedence tests.
2022-11-26 17:02:37 +01:00
Darren Schroeder
3e76ed9122
add into record command (#7225)
# Description

This command converts things into records.
<img width="466" alt="Screenshot 2022-11-24 at 2 10 54 PM"
src="https://user-images.githubusercontent.com/343840/203858104-0e4445da-9c37-4c7c-97ec-68ec3515bc4b.png">

<img width="716" alt="Screenshot 2022-11-24 at 5 04 11 PM"
src="https://user-images.githubusercontent.com/343840/203872621-48cab199-ba57-44fe-8f36-9e1469b9c4ef.png">



It also converts dates into record but I couldn't get the test harness
to accept an example.

Thanks to @WindSoilder for writing the "hard" parts of this. :)


_(Thank you for improving Nushell. Please, check our [contributing
guide](../CONTRIBUTING.md) and talk to the core team before making major
changes.)_

_(Description of your pull request goes here. **Provide examples and/or
screenshots** if your changes affect the user experience.)_

# User-Facing Changes

_(List of all changes that impact the user experience here. This helps
us keep track of breaking changes.)_

# Tests + Formatting

Don't forget to add tests that cover your changes.

Make sure you've run and fixed any issues with these commands:

- `cargo fmt --all -- --check` to check standard code formatting (`cargo
fmt --all` applies these changes)
- `cargo clippy --workspace -- -D warnings -D clippy::unwrap_used -A
clippy::needless_collect` to check that you're using the standard code
style
- `cargo test --workspace` to check that all tests pass

# After Submitting

If your PR had any user-facing changes, update [the
documentation](https://github.com/nushell/nushell.github.io) after the
PR is merged, if necessary. This will help us keep the docs up to date.

Co-authored-by: WindSoilder <WindSoilder@outlook.com>
2022-11-26 09:00:47 -06:00
JT
74a73f9838
Stdout/Stderr redirection (#7185)
This adds new pipeline connectors called out> and err> which redirect either stdout or stderr to a file. You can also use out+err> (or err+out>) to redirect both streams into a file.
2022-11-23 07:26:13 +13:00
Leon
4b83a2d27a
Improve CantFindColumn and ColumnAlreadyExists errors (#7164)
* Improve CantFindColumn and ColumnAlreadyExists errors

* Update tests
2022-11-19 09:35:55 -08:00
JT
13515c5eb0
Limited mutable variables (#7089)
This adds support for (limited) mutable variables. Mutable variables are created with mut much the same way immutable variables are made with let.

Mutable variables allow mutation via the assignment operator (=).

❯ mut x = 100
❯ $x = 200
❯ print $x
200

Mutable variables are limited in that they're only tended to be used in the local code block. Trying to capture a local variable will result in an error:

❯ mut x = 123; {|| $x }
Error: nu::parser::expected_keyword (link)

  × Capture of mutable variable.

The intent of this limitation is to reduce some of the issues with mutable variables in general: namely they make code that's harder to reason about. By reducing the scope that a mutable variable can be used it, we can help create local reasoning about them.

Mutation can occur with fields as well, as in this case:

❯ mut y = {abc: 123}
❯ $y.abc = 456
❯ $y

On a historical note: mutable variables are something that we resisted for quite a long time, leaning as much as we could on the functional style of pipelines and dataflow. That said, we've watched folks struggle to work with reduce as an approximation for patterns that would be trivial to express with local mutation. With that in mind, we're leaning towards the happy path.
2022-11-11 19:51:08 +13:00
David Matos
312e9bf5d6
fix overflow on negative bytes (#7070) 2022-11-10 22:33:15 +01:00
JT
63433f1bc8
Split blocks and closures (#7075)
* Split closures and blocks

* Tests mostly working

* finish last fixes, passes all tests

* fmt
2022-11-10 21:21:49 +13:00
Leon
921a66554e
Replace all instances of 'column path' in help messages with 'cell path' (#7063)
* Rewrite all 'column path' instances to 'cell path'

* Minor tweak
2022-11-09 21:49:11 -08:00
Dan Davison
df94052180
Declare input and output types of commands (#6796)
* Add failing test that list of ints and floats is List<Number>

* Start defining subtype relation

* Make it possible to declare input and output types for commands

- Enforce them in tests

* Declare input and output types of commands

* Add formatted signatures to `help commands` table

* Revert SyntaxShape::Table -> Type::Table change

* Revert unnecessary derive(Hash) on SyntaxShape

Co-authored-by: JT <547158+jntrnr@users.noreply.github.com>
2022-11-10 10:55:05 +13:00
Alex Saveau
be5d71ea47
Run a round of clippy --fix to fix a ton of lints (#7006)
Signed-off-by: Alex Saveau <saveau.alexandre@gmail.com>

Signed-off-by: Alex Saveau <saveau.alexandre@gmail.com>
2022-11-04 15:11:17 -05:00
WindSoilder
4f7f6a2932
Friendly error message for access beyond end (#6944)
Adds `ShellError::AccessEmptyContent`
2022-10-29 19:47:50 +02:00
Reilly Wood
e0cc2c9112
Make get 1 error message better (#6892) 2022-10-24 18:22:57 -07:00
Reilly Wood
17b2bcc125
Support range in str substring (#6867) 2022-10-23 11:42:17 +02:00
pwygab
5e748ae8fc
make ++ append lists (#6766)
* make `++` append lists

* fmt

* fix for database
2022-10-20 23:28:18 +13:00
pwygab
f5e1b08e6a
ensure Operator::And errors out with incompatible types (#6638) 2022-09-29 06:17:21 -05:00
Darren Schroeder
dd578926c3
add some float operations with filesize (#6618)
* add some float operations with filesize

* more changes

* update return values on filesize-filesize, duration-duration

* missed filesize in floordiv

* missed float * duration
2022-09-28 17:07:50 -05:00
JT
d66a5398d1
Remove month and year duration constants (#6613)
remove month/year/decade durations for parsing and units, but leave the humanized output for viewing
2022-09-26 09:55:13 +13:00
Dan Davison
367f79cb4f
Don't compute 'did you mean' suggestions unless showing them to user (#6540) 2022-09-11 09:58:19 -07:00
pwygab
e81689f2c0
Allow for rejecting nested record cells (#6463)
* add new function to remove data at a cellpath; allow reject to use cellpath

* add tests

* fmt

* fix clippt

* get it working properly with lists of records

* fix clippy, hopefully

* fix clippy, hopefully 2
2022-09-03 07:35:36 -05:00
Darren Schroeder
4858a9a817
Revert "Add support for optional list stream output formatting (#6325)" (#6454)
This reverts commit ec4e3a6d5c.
2022-08-31 18:09:40 -05:00
panicbit
ec4e3a6d5c
Add support for optional list stream output formatting (#6325)
* add support for optional list stream output formatting

* cargo fmt

* table: add ValueFormatter test
2022-08-18 05:44:53 -05:00
Björn Richter
cb18dd5200
Add decimals to int when using into string --decimals (#6085)
* Add decimals to int when using `into string --decimals`

* Add tests for `into string` when converting int with `--decimals`

* Apply formatting

* Merge `into_str` test files

* Comment out unused code and add TODOs

* Use decimal separator depending on system locale

* Add test helper to run closure in different locale

* Add tests for int-to-string conversion using different locales

* Add utils function to get system locale

* Add panic message when locking mutex fails

* Catch and resume panic later to prevent Mutex poisoning when test fails

* Move test to `nu-test-support` to keep `nu-utils` free of `nu-*` dependencies

See https://github.com/nushell/nushell/pull/6085#issuecomment-1193131694

* Rename test support fn `with_fake_locale` to `with_locale_override`

* Move `get_system_locale()` to `locale` module

* Allow overriding locale with special env variable (when not in release)

* Use special env var to override locale during testing

* Allow callback to return a value in `with_locale_override()`

* Allow multiple options in `nu!` macro

* Allow to set locale as `nu!` macro option

* Use new `locale` option of `nu!` macro instead of `with_locale_override`

Using the `locale` options does not lock the `LOCALE_OVERRIDE_MUTEX`
mutex in `nu-test-support::locale_override` but instead calls the `nu`
command directly with the `NU_LOCALE_OVERRIDE` environment variable.
This allows for parallel test excecution.

* Fix: Add option identifier for `cwd` in usage of `nu!` macro

* Rely on `Display` trait for formatting `nu!` macro command

- Removed the `DisplayPath` trait
- Implement `Display` for `AbsolutePath`, `RelativePath` and
  `AbsoluteFile`

* Default to locale `en_US.UTF-8` for tests when using `nu!` macro

* Add doc comment to `nu!` macro

* Format code using `cargo fmt --all`

* Pass function directly instead of wrapping the call in a closure

https://rust-lang.github.io/rust-clippy/master/index.html#redundant_closure

* Pass function to `or_else()` instead of calling it inside `or()`

https://rust-lang.github.io/rust-clippy/master/index.html#or_fun_call

* Fix: Add option identifier for `cwd` in usage of `nu!` macro
2022-08-12 21:13:50 -05:00
Darren Schroeder
cdeb8de75d
replace the regex crate with the fancy-regex crate (#6227) 2022-08-04 14:51:02 -05:00
Darren Schroeder
d856ac92f4
expand durations to include month, year, decade (#6123)
* expand durations to include month, year, decade

* remove commented out fn

* oops, found more debug comments

* tweaked tests for the new way, borrowed heavily from chrono-humanize-rs

* clippy

* grammar
2022-07-26 08:05:37 -05:00
Jakub Žádník
f3036b8cfd
Allow keeping selected environment variables from removed overlay (#6007)
* Allow keeping selected env from removed overlay

* Remove some duplicate code

* Change --keep-all back to --keep-custom

Because, apparently, you cannot have a named flag called --keep-all,
otherwise tests fail?

* Fix missing line and wrong test value
2022-07-11 23:58:28 +03:00
WindSoilder
9e3c64aa84
Add bytes collect, bytes remove, bytes build cmd (#6008)
* add bytes collect

* index_of support searching from end

* add bytes remove

* make bytes replace work better for empty pattern

* add bytes build

* remove comment

* tweak words

Co-authored-by: Darren Schroeder <343840+fdncred@users.noreply.github.com>
2022-07-11 06:26:00 -05:00
Justin Ma
4e90b478b7
Add bit operator: bit-xor (#5940) 2022-07-03 06:45:20 -05:00
JT
a48616697a
Rename bitwise operators for readability (#5937) 2022-07-02 17:05:02 -05:00
Justin Ma
b82dccf0bd
Add band and bor operator for bit operations (#5936)
* Add `band` and `bor` Operator

* Add tests
2022-07-02 13:03:36 -05:00
Justin Ma
3917fda7ed
Update #4202: Add shift operator bshl and bshr for integers (#5928)
* Update #4202: Add shift operator bshl and bshr for integers

* Add more tests
2022-07-02 06:48:43 -05:00
Benoît Cortier
173d60d59d
Deprecate hash base64, extend decode and add encode commands (#5863)
* feat: deprecate `hash base64` command

* feat: extend `decode` and `encode` command families

This commit
- Adds `encode` command family
- Backports `hash base64` features to `encode base64` and `decode base64` subcommands.
- Refactors code a bit and extends tests for encodings
- `decode base64` returns a binary `Value` (that may be decoded into a string using `decode` command)

* feat: add `--binary(-b)` flag to `decode base64`

Default output type is now string, but binary can be requested using this new flag.
2022-06-26 00:35:23 +03:00
Fernando Herrera
de554f8e5f
filesize conversion (#5770) 2022-06-13 14:44:32 -05:00
pwygab
caafd26deb
Attempts to add // math operator (#5759)
* attempts to add `div` math operator

* allows `//` to be used too

* fmt:

* clippy issue

* returns appropriate type

* returns appropriate type 2

* fmt

* ensure consistency; rename to `fdiv`

* Update parser.rs
2022-06-13 13:54:47 +03:00
Fernando Herrera
d5b99ae316
input and output types (#5750)
* input and output types

* added description

* type from stored variable

* string in custom value

* more tests with non custom
2022-06-10 10:59:35 -05:00
Darren Schroeder
c57f41e5f2
make to text work more intuitively (#5733) 2022-06-07 14:43:24 -05:00
Stefan Holderbach
e5d38dcff6
Address lints from clippy for beta/nightly (#5709)
* Fix clippy lints in tests

* Replace `format!` in `.push_str()` with `write!`

Stylistically that might be a bit rough but elides an allocation.

Fallibility of allocation is more explicit, but ignored with `let _ =`
like in the clippy example:

https://rust-lang.github.io/rust-clippy/master/index.html#format_push_string

* Remove unused lifetime

* Fix macro crate relative import

* Derive `Eq` for `PartialEq` with `Eq` members

https://rust-lang.github.io/rust-clippy/master/index.html#derive_partial_eq_without_eq

* Remove unnnecessary `.to_string()` for Cow<str>

* Remove `.to_string()` for `tendril::Tendril`

Implements `Deref<Target = str>`
2022-06-04 18:47:36 +12:00
Justin Ma
d44059c36b
feat: Add sensitive flag to get, fix #4295 (#5685)
* feat: Add insensitive flag to get, fix #4295

* add get insensitive example

* Fix get flags

* Update get examples
2022-06-01 08:34:42 -05:00
WindSoilder
fc41a0f96b
use reverse iter on value search (#5553) 2022-05-16 06:29:40 -05:00
Fernando Herrera
8bd68416e3
Lazy dataframes (#5546)
* lazyframe definition

* expressions and lazy frames

* new alias expression

* more expression commands

* updated to polars main

* more expressions and groupby

* more expressions, fetch and sort-by

* csv reader

* removed open csv

* unique function

* joining functions

* join lazy frames commands with eager commands

* corrected tests

* Update .gitignore

* Update .gitignore

Co-authored-by: JT <547158+jntrnr@users.noreply.github.com>
2022-05-16 08:27:43 +01:00
WindSoilder
16bd7b6d0d
Fix Value::Record compare logic, and pass uniq tests. (#5541)
* fix record compare logic

* add more comment
2022-05-14 06:04:09 -05:00
WindSoilder
8030f7e9f0
add format filesize (#5498)
* add format filesize

* add comment

* add comment

* remove comment
2022-05-10 06:35:14 -05:00
panicbit
49cbc30974
Add ends-with operator and fix dataframe operator behavior (#5395)
* add ends-with operator

* escape needles in dataframe operator regex patterns
2022-05-02 20:02:38 +12:00
Andrés N. Robalino
7a7aa310aa
Remove 'empty' block support reminders, for now. (#5214) 2022-04-30 22:32:30 -05:00
Reilly Wood
b501db673a
SQLite overhaul: custom value, query db command (#5247)
Clean up query errors
2022-04-20 16:58:21 +12:00
Kat Marchán
1314a87cb0
update miette and switch to GenericErrors (#5222) 2022-04-19 00:34:10 +12:00
JT
14066ccc30
Fix known externals, fix operator spans (#5140) 2022-04-09 17:17:48 +12:00
Michael Angerman
aaec840b91
doc change from engine-q to nushell (#5134) 2022-04-08 10:29:21 -07:00
Reilly Wood
b2c52b51b7
Change string contains operators to regex (#5117) 2022-04-07 18:23:14 +12:00
JT
4409185e1b
Improve describe to be more accurate (#5116) 2022-04-07 16:34:09 +12:00
Darren Schroeder
2cb815b7b4
Add starts with operator (#5061)
* add starts_with operator

* added a test
2022-04-01 13:35:46 -05:00
JT
2e3b74f1b2
Fix for loop ctrlc not terminating (#5003) 2022-03-28 19:13:43 +13:00
Jonathan Moore
ea7c8c237e
CantConvert improvements (#4926)
* CantConvert improvements

* cargo fmt
2022-03-24 07:04:31 -05:00
JT
0986eefb64
Add insert and update back (#4864) 2022-03-18 06:55:02 +13:00
JT
088d19ad47
Make date values more readable (#4790) 2022-03-09 05:43:04 -05:00
JT
1837bf775c
Default values (#4770) 2022-03-07 15:08:56 -05:00
JT
cb5c61d217
Fix open ended ranges (#4677)
* Make open ended ranges more open ended

* Add test
2022-02-28 11:15:31 -05:00
Fernando Herrera
11bc056576
Find with regex flag (#4649)
* split find functions

* find command with regex

* corrected message

* cargo fmt
2022-02-26 04:19:19 -05:00
JT
3c62d27c28
Try again with math-like externals (#4629)
* Try again with math-like externals

* clippy 1.59

* clippy 1.59

* clippy 1.59
2022-02-24 14:02:28 -05:00
Michael Angerman
feb4f5c347
replace ValueStream with ListStream (#4621) 2022-02-24 06:57:31 -05:00
Fernando Herrera
21c0f7d738
allow int and float as strings for arguments (#4615)
* allow int and float as strings for arguments

* consume iterator
2022-02-24 05:09:02 +00:00
JT
4b18fdcc6e
Date literals (#4619)
* Date literals

* update deps

* Add date+duration
2022-02-23 21:02:48 -05:00
JT
2ba12afb01
A few fixes to docs generation and default config (#4570)
* A few fixes to docs generation and default config

* A few more fixes
2022-02-20 15:20:41 -05:00
JT
d53eaac7a1
Improve comparison errors (#4541) 2022-02-18 17:11:27 -05:00
JT
06f9047be4
Add an explicit 'print' command (#4535) 2022-02-18 13:43:34 -05:00
JT
786e4ab971
Make 'for' implicitly filter out nothings (#4536)
* Make 'for' implicitly filter out nothings

* Fix test
2022-02-18 13:41:41 -05:00
JT
d620f76a21
Make comparisons/sort-by more 'global' (#4500)
* Make comparisons/sort-by more 'global'

* Let custom values do their own comparisons
2022-02-16 13:30:37 -06:00
JT
c4e1559f89
Another batch of command tests (#4496)
* Add a batch of command tests

* More tests
2022-02-16 07:38:02 -05:00
JT
5a1d81221f
Move 'nth' into 'select' (#4385) 2022-02-09 09:59:40 -05:00
JT
d70d91e559 Remove old nushell/merge engine-q 2022-02-07 14:54:06 -05:00
Fernando Herrera
fdce6c49ab engine-q merge 2022-02-07 19:11:34 +00:00
JT
a78c82d811
Make PipelineData helpers collect rawstreams (#969) 2022-02-07 07:44:18 -05:00
JT
cc1b784e3d
Add initial nu-test-support port (#913)
* Add initial nu-test-support port

* finish changing binary name

* Oops, these aren't Windows-safe tests
2022-02-02 15:59:01 -05:00
Access
1fd0ddb52c
Maybe solve the none bug? (#860)
* Maybe solve the none bug?

* cargo fmt

* use nothing, not string

* check at last

* I check it at last

* Use error which has span

* use not found error

* fix error

* use a empty value length?

* * Add commit about what I change and fmt

Now all test passed, but I do not know if it is right

* update the test

* check if it is nothing

* update commit

* Rename test

Co-authored-by: Jakub Žádník <kubouch@gmail.com>
2022-01-30 15:23:28 +02:00
JT
e91d8655c6
Only trim prompt (#876)
* Only trim the output for prompts

* Only remove the last newline
2022-01-28 18:22:09 -05:00
JT
4c029d2545
Automatically trim ends of stdin/stdout strings (#874) 2022-01-28 16:59:00 -05:00
JT
86eeb4a5e7
Fix a bad slice into erroring utf-8 buffer (#872) 2022-01-28 15:32:46 -05:00
JT
020ad24b25
"maybe text codec" version 2 (#871)
* Add a RawStream that can be binary or string

* Finish up updating the into's
2022-01-28 13:32:33 -05:00
JT
6e44012a2f
Fix bug in date comparison (#842) 2022-01-24 16:55:45 -05:00
JT
988a873466
Allow open to read its filename from input (#841)
* Allow `open` to read its filename from input

* Add examples
2022-01-25 08:04:28 +11:00
Arthur Targaryen
f82e2fbac6
Port find command (#658)
* Add `Find` command

* Complete rustdoc for test `Value` constructors

* Use `Option::unwrap_or` instead of match

* Add `Value::test_filesize` constructor

* Handle searching for terms in `find`

* Fix `find` command signature

* Return multiple elements when `find`ing by predicate

* Do not accept rest parameter with predicate

* Handle `CellPath` in `r#in` and `not_in` for `Find`

* Use `PipelineData::filter`
2022-01-23 16:32:02 -06:00
Jakub Žádník
564c2dd7d1
Port merge command from Nushell (#808)
* Add example test to zip

* Port merge command from Nushell

On top of the original merge, this one should not collect a stream
returned from the merged block and allows merging records.
2022-01-22 01:50:26 +02:00
JT
45b3592739
add some more division for units (#783) 2022-01-21 05:23:26 +11:00
Fernando Herrera
6514a30b5d
general keybindings (#775)
* general keybindings

* get value function

* check error for keybinding

* cmd and send for keybingins

* better error message
2022-01-19 07:28:08 -06:00
Darren Schroeder
2c75aabbfc
allow size and other to count bytes from binary with as_string() (#769) 2022-01-17 17:41:59 -06:00
Jakub Žádník
3b4baa31b6
Fix ls relative path & command argument path expansion (#757)
* Switch to short-names when the path is a relative_path (a dir) and exit with an error if the path does not exist

* Remove debugging print line

* Show relative filenames... It does not work yet for ls ../

* Try something else to fix relative paths... it works, but the ../ code part is not very pretty

* Add canonicalize check and remove code clones

* Fix the canonicalize_with issue pointed out by kubouch. Not sure the prefix_str is what kubouch suggested

* Fix the canonicalize_with issue pointed out by kubouch. Not sure the prefix_str is what kubouch suggested

* Add single-dot expansion to nu-path

* Move value path expansion from parser to eval

Fixes #745

* Remove single dot expansion from parser

It is not necessary since it will get expanded anyway in the eval.

* Fix ls to display globs with relative paths

* Use pathdiff crate to get relative paths for ls

Co-authored-by: Stefan Stanciulescu <contact@stefanstanciulescu.com>
2022-01-17 00:55:56 +11:00
JT
b78924c777
Add support for load-env (#752) 2022-01-15 18:50:11 -05:00