# Description This grew quite a bit beyond its original scope, but I've tried to make `$in` a bit more consistent and easier to work with. Instead of the parser generating calls to `collect` and creating closures, this adds `Expr::Collect` which just evaluates in the same scope and doesn't require any closure. When `$in` is detected in an expression, it is replaced with a new variable (also called `$in`) and wrapped in `Expr::Collect`. During eval, this expression is evaluated directly, with the input and with that new variable set to the collected value. Other than being faster and less prone to gotchas, it also makes it possible to typecheck the output of an expression containing `$in`, which is nice. This is a breaking change though, because of the lack of the closure and because now typechecking will actually happen. Also, I haven't attempted to typecheck the input yet. The IR generated now just looks like this: ```gas collect %in clone %tmp, %in store-variable $in, %tmp # %out <- ...expression... <- %in drop-variable $in ``` (where `$in` is the local variable created for this collection, and not `IN_VARIABLE_ID`) which is a lot better than having to create a closure and call `collect --keep-env`, dealing with all of the capture gathering and allocation that entails. Ideally we can also detect whether that input is actually needed, so maybe we don't have to clone, but I haven't tried to do that yet. Theoretically now that the variable is a unique one every time, it should be possible to give it a type - I just don't know how to determine that yet. On top of that, I've also reworked how `$in` works in pipeline-initial position. Previously, it was a little bit inconsistent. For example, this worked: ```nushell > 3 | do { let x = $in; let y = $in; print $x $y } 3 3 ``` However, this causes a runtime variable not found error on the second `$in`: ```nushell > def foo [] { let x = $in; let y = $in; print $x $y }; 3 | foo Error: nu:🐚:variable_not_found × Variable not found ╭─[entry #115:1:35] 1 │ def foo [] { let x = $in; let y = $in; print $x $y }; 3 | foo · ─┬─ · ╰── variable not found ╰──── ``` I've fixed this by making the first element `$in` detection *always* happen at the block level, so if you use `$in` in pipeline-initial position anywhere in a block, it will collect with an implicit subexpression around the whole thing, and you can then use that `$in` more than once. In doing this I also rewrote `parse_pipeline()` and hopefully it's a bit more straightforward and possibly more efficient too now. Finally, I've tried to make `let` and `mut` a lot more straightforward with how they handle the rest of the pipeline, and using a redirection with `let`/`mut` now does what you'd expect if you assume that they consume the whole pipeline - the redirection is just processed as normal. These both work now: ```nushell let x = ^foo err> err.txt let y = ^foo out+err>| str length ``` It was previously possible to accomplish this with a subexpression, but it just seemed like a weird gotcha that you couldn't do it. Intuitively, `let` and `mut` just seem to take the whole line. - closes #13137 # User-Facing Changes - `$in` will behave more consistently with blocks and closures, since the entire block is now just wrapped to handle it if it appears in the first pipeline element - `$in` no longer creates a closure, so what can be done within an expression containing `$in` is less restrictive - `$in` containing expressions are now type checked, rather than just resulting in `any`. However, `$in` itself is still `any`, so this isn't quite perfect yet - Redirections are now allowed in `let` and `mut` and behave pretty much how you'd expect # Tests + Formatting Added tests to cover the new behaviour. # After Submitting - [ ] release notes (definitely breaking change)
163 lines
5.6 KiB
Rust
163 lines
5.6 KiB
Rust
use chrono::FixedOffset;
|
|
use serde::{Deserialize, Serialize};
|
|
|
|
use super::{
|
|
Call, CellPath, Expression, ExternalArgument, FullCellPath, Keyword, MatchPattern, Operator,
|
|
Range, Table, ValueWithUnit,
|
|
};
|
|
use crate::{
|
|
ast::ImportPattern, engine::StateWorkingSet, BlockId, OutDest, Signature, Span, VarId,
|
|
};
|
|
|
|
/// An [`Expression`] AST node
|
|
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
|
|
pub enum Expr {
|
|
Bool(bool),
|
|
Int(i64),
|
|
Float(f64),
|
|
Binary(Vec<u8>),
|
|
Range(Box<Range>),
|
|
Var(VarId),
|
|
VarDecl(VarId),
|
|
Call(Box<Call>),
|
|
ExternalCall(Box<Expression>, Box<[ExternalArgument]>), // head, args
|
|
Operator(Operator),
|
|
RowCondition(BlockId),
|
|
UnaryNot(Box<Expression>),
|
|
BinaryOp(Box<Expression>, Box<Expression>, Box<Expression>), //lhs, op, rhs
|
|
Collect(VarId, Box<Expression>),
|
|
Subexpression(BlockId),
|
|
Block(BlockId),
|
|
Closure(BlockId),
|
|
MatchBlock(Vec<(MatchPattern, Expression)>),
|
|
List(Vec<ListItem>),
|
|
Table(Table),
|
|
Record(Vec<RecordItem>),
|
|
Keyword(Box<Keyword>),
|
|
ValueWithUnit(Box<ValueWithUnit>),
|
|
DateTime(chrono::DateTime<FixedOffset>),
|
|
/// The boolean is `true` if the string is quoted.
|
|
Filepath(String, bool),
|
|
/// The boolean is `true` if the string is quoted.
|
|
Directory(String, bool),
|
|
/// The boolean is `true` if the string is quoted.
|
|
GlobPattern(String, bool),
|
|
String(String),
|
|
RawString(String),
|
|
CellPath(CellPath),
|
|
FullCellPath(Box<FullCellPath>),
|
|
ImportPattern(Box<ImportPattern>),
|
|
Overlay(Option<BlockId>), // block ID of the overlay's origin module
|
|
Signature(Box<Signature>),
|
|
StringInterpolation(Vec<Expression>),
|
|
/// The boolean is `true` if the string is quoted.
|
|
GlobInterpolation(Vec<Expression>, bool),
|
|
Nothing,
|
|
Garbage,
|
|
}
|
|
|
|
// This is to document/enforce the size of `Expr` in bytes.
|
|
// We should try to avoid increasing the size of `Expr`,
|
|
// and PRs that do so will have to change the number below so that it's noted in review.
|
|
const _: () = assert!(std::mem::size_of::<Expr>() <= 40);
|
|
|
|
impl Expr {
|
|
pub fn pipe_redirection(
|
|
&self,
|
|
working_set: &StateWorkingSet,
|
|
) -> (Option<OutDest>, Option<OutDest>) {
|
|
match self {
|
|
Expr::Call(call) => working_set.get_decl(call.decl_id).pipe_redirection(),
|
|
Expr::Collect(_, _) => {
|
|
// A collect expression always has default redirection, it's just going to collect
|
|
// stdout unless another type of redirection is specified
|
|
(None, None)
|
|
},
|
|
Expr::Subexpression(block_id) | Expr::Block(block_id) => working_set
|
|
.get_block(*block_id)
|
|
.pipe_redirection(working_set),
|
|
Expr::FullCellPath(cell_path) => cell_path.head.expr.pipe_redirection(working_set),
|
|
Expr::Bool(_)
|
|
| Expr::Int(_)
|
|
| Expr::Float(_)
|
|
| Expr::Binary(_)
|
|
| Expr::Range(_)
|
|
| Expr::Var(_)
|
|
| Expr::UnaryNot(_)
|
|
| Expr::BinaryOp(_, _, _)
|
|
| Expr::Closure(_) // piping into a closure value, not into a closure call
|
|
| Expr::List(_)
|
|
| Expr::Table(_)
|
|
| Expr::Record(_)
|
|
| Expr::ValueWithUnit(_)
|
|
| Expr::DateTime(_)
|
|
| Expr::String(_)
|
|
| Expr::RawString(_)
|
|
| Expr::CellPath(_)
|
|
| Expr::StringInterpolation(_)
|
|
| Expr::GlobInterpolation(_, _)
|
|
| Expr::Nothing => {
|
|
// These expressions do not use the output of the pipeline in any meaningful way,
|
|
// so we can discard the previous output by redirecting it to `Null`.
|
|
(Some(OutDest::Null), None)
|
|
}
|
|
Expr::VarDecl(_)
|
|
| Expr::Operator(_)
|
|
| Expr::Filepath(_, _)
|
|
| Expr::Directory(_, _)
|
|
| Expr::GlobPattern(_, _)
|
|
| Expr::ImportPattern(_)
|
|
| Expr::Overlay(_)
|
|
| Expr::Signature(_)
|
|
| Expr::Garbage => {
|
|
// These should be impossible to pipe to,
|
|
// but even it is, the pipeline output is not used in any way.
|
|
(Some(OutDest::Null), None)
|
|
}
|
|
Expr::RowCondition(_) | Expr::MatchBlock(_) => {
|
|
// These should be impossible to pipe to,
|
|
// but if they are, then the pipeline output could be used.
|
|
(None, None)
|
|
}
|
|
Expr::ExternalCall(_, _) => {
|
|
// No override necessary, pipes will always be created in eval
|
|
(None, None)
|
|
}
|
|
Expr::Keyword(_) => {
|
|
// Not sure about this; let's return no redirection override for now.
|
|
(None, None)
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
/// Expressions permitted inside a record expression/literal
|
|
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
|
|
pub enum RecordItem {
|
|
/// A key: val mapping
|
|
Pair(Expression, Expression),
|
|
/// Span for the "..." and the expression that's being spread
|
|
Spread(Span, Expression),
|
|
}
|
|
|
|
/// Expressions permitted inside a list expression/literal
|
|
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
|
|
pub enum ListItem {
|
|
/// A normal expression
|
|
Item(Expression),
|
|
/// Span for the "..." and the expression that's being spread
|
|
Spread(Span, Expression),
|
|
}
|
|
|
|
impl ListItem {
|
|
pub fn expr(&self) -> &Expression {
|
|
let (ListItem::Item(expr) | ListItem::Spread(_, expr)) = self;
|
|
expr
|
|
}
|
|
|
|
pub fn expr_mut(&mut self) -> &mut Expression {
|
|
let (ListItem::Item(expr) | ListItem::Spread(_, expr)) = self;
|
|
expr
|
|
}
|
|
}
|