Support partially clearing messages with FieldMask?

[tomhoule]

The docs for FieldMask (from the well-known types) describe how it can be used for selectively clearing fields on a message. This could be implemented using a custom derive.

Since I would like to use this feature I may be interested in making a PR for implementing it. It could either be part of the generated impls for the messages, taking an Iterator<Item=&str> to avoid depending on prost-types, or implemented there only as an optional trait that can be added in build scripts (EDIT: having it be outside prost is also an option).

Is this a desirable feature, and does this sound reasonable?

[danburkert]

Interesting, this is one corner of the protobuf feature set I've never encountered. Could you elaborate a bit on when this comes in handy? If it's a useful feature then I'd be positive on adding it to prost. From looking at the C++ docs on the feature, it seems like it's implemented using reflection, which prost is currently lacking. Implementing via derive macro may be possible as well. Were you thinking of implementing the masked merge functionality as well?

[tomhoule]

The use-case in my mind was something a bit analogous to GraphQL queries: it makes evolving an API easier (because the client has to request explicitly all the fields it needs, and the server knows which ones are no longer used), and it can diminish response payload sizes.

I went with GraphQL incidentally with the current project, so I won't be able to make time to implement this, but I think as a derive it shouldn't be too complicated (the only non-trivial part is the nesting).

The masked merge has a different use-case (and I don't think much can be shared in terms of implementation), I'm not a heavy user of protocol buffers at the moment but it sounds like a cool feature to have.

[allancalix]

I may be missing something but is there a way to set fields based on string paths? Something like the extended merge provided in java that applies field masks would be a good addition:

https://developers.google.com/protocol-buffers/docs/reference/java/com/google/protobuf/util/FieldMaskUtil

[frederikhors]

@tomhoule nothing changed about this, right? I can't understand why no one has thought of it yet and how they do without it!

[gonzalezzfelipe]

There's probably a million better ways of doing this... But I'm posting this here because I solved it like this and it might help someone else with it. AFAIK it works, might be a good starting point to implement something more stable

use serde::{de::DeserializeOwned, Serialize};
use serde_json::{json, Value};

fn get_nested_value(value: &Value, keys: &[&str]) -> Option<Value> {
    let mut current = value;
    for key in keys {
        match current {
            Value::Object(map) => current = map.get(*key)?,
            _ => return None,
        }
    }
    Some(current.clone())
}

fn set_nested_value(value: &mut Value, keys: &[&str], new_value: Value) {
    let mut current = value;
    for key in keys.iter().take(keys.len() - 1) {
        current = current
            .as_object_mut()
            .unwrap() // Safe to unwrap when used in conjunction with get_nested_value
            .entry(key.to_string())
            .or_insert_with(|| json!({}));
    }
    current
        .as_object_mut()
        .unwrap()
        .insert(keys.last().unwrap().to_string(), new_value);
}

pub fn apply_mask<T>(obj: T, paths: Vec<String>) -> Result<T, serde_json::Error>
where
    T: DeserializeOwned + Serialize,
{
    let json_value = serde_json::to_value(obj)?;
    // Create a new JSON object with only the specified paths
    let mut new_json = json!({});

    for path in paths {
        let keys: Vec<&str> = path.split('.').collect();
        if let Some(value) = get_nested_value(&json_value, &keys) {
            set_nested_value(&mut new_json, &keys, value);
        }
    }
    serde_json::from_value::<T>(new_json)
}

#[cfg(test)]
mod tests {
    use serde::Deserialize;

    use super::*;

    #[derive(Deserialize, Serialize, Default, Debug, PartialEq, Clone)]
    #[serde(default)]
    struct Bar {
        pub abc: usize,
    }

    #[derive(Deserialize, Serialize, Default, Debug, Clone)]
    #[serde(default)]
    struct Foo {
        pub foo: String,
        pub bar: Bar,
    }

    #[test]
    fn test_apply_mask() {
        let foobar = Foo {
            foo: "foo".into(),
            bar: Bar { abc: 10 },
        };
        let paths = vec!["foo".to_string()];
        let masked_foobar = apply_mask(foobar.clone(), paths).expect("Failed to serde");
        assert_eq!(masked_foobar.foo, "foo");
        assert_eq!(masked_foobar.bar, Bar::default());

        let paths = vec!["bar".to_string()];
        let masked_foobar = apply_mask(foobar, paths).expect("Failed to serde");
        assert_eq!(masked_foobar.foo, String::default());
        assert_eq!(masked_foobar.bar.abc, 10);
    }
}