Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ability to change language without a restart #14379

Closed

Conversation

rubenwardy
Copy link
Contributor

@rubenwardy rubenwardy commented Feb 17, 2024

This allows changing the current language without restarting Minetest. Gettext reads the language env var for every call, so we just need to change this and notify gettext about the change

To do

This PR is Ready for Review.

How to test

  • Set system language to non-English, ie French
  • Clear minetest.conf
  • Open Minetest, see in French language
  • Change Minetest language setting to German. See it change to German
  • Change Minetest language setting back to (Use system language). See French

@y5nw
Copy link
Contributor

y5nw commented Feb 17, 2024

Two minor issues I found:

  1. Switching from a specific language to "[Use system language]" does not seem to change the language.
  2. (Probably nitpicking here:) Changing the language when MT is run using LC_ALL=de_DE.UTF-8 bin/minetest (or LANG=...) messes up the UI (see attachment); I can only reproduce this with LC_ALL=de_DE.UTF-8 though, for some reason

2024-02-17-18-56-02

@rubenwardy rubenwardy added the Action / change needed Code still needs changes (PR) / more information requested (Issues) label Feb 18, 2024
// Source: https://www.gnu.org/software/gettext/manual/html_node/gettext-grok.html
{
extern int _nl_msg_cat_cntr;
++_nl_msg_cat_cntr;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if this won't mess up on systems where they are using some sort of alternative gettext implementation

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whilst the documentation says to do this, I found that it wasn't actually needed. So if it's likely to cause issues, could maybe be removed

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Related: https://bugs.kde.org/show_bug.cgi?id=365917 (fixed by https://invent.kde.org/frameworks/ki18n/-/commit/543b8ae67c57500bf3cabde75bf1599904d6e482) and also the #14379 (comment) comment below.

Maybe only enable language switching when it can be determined that it works? (e.g. by checking whether translating LANG_CODE works after switching languages)

src/gettext.cpp Show resolved Hide resolved
@grorp
Copy link
Member

grorp commented Feb 21, 2024

What do you think about using a SettingsChangedCallback instead of manually reporting changes via core.set_language? This way, I imagine that changing the setting via a chat command in singleplayer (e.g. /set language it) could also work.

@rubenwardy
Copy link
Contributor Author

rubenwardy commented Feb 21, 2024

What do you think about using a SettingsChangedCallback instead of manually reporting changes via core.set_language? This way, I imagine that changing the setting via a chat command in singleplayer (e.g. /set language it) could also work.

Yeah I was thinking about this but went with the simpler option to begin with

One thing to point out is that /set does the server-side setting, so may only work in singleplayer (assuming set_gettext_language is thread safe). I don't know if there's a client-side equivalent to /set (.set ?). In any case, having a language setting in the pause menu (#6722) would be very welcome

@rubenwardy rubenwardy removed the Action / change needed Code still needs changes (PR) / more information requested (Issues) label Feb 21, 2024
@rubenwardy
Copy link
Contributor Author

Updated, please may you retest @y5nw

src/gettext.cpp Outdated Show resolved Hide resolved
@rubenwardy rubenwardy added Action / change needed Code still needs changes (PR) / more information requested (Issues) and removed Action / change needed Code still needs changes (PR) / more information requested (Issues) labels Feb 24, 2024
@rubenwardy
Copy link
Contributor Author

rubenwardy commented Feb 25, 2024

Ok, updated. It works as expected for me with LANGUAGE, although I'm unable to test LC_ALL

This will need to be tested on MSVC stuff, as I've changed code gated by _MSC_VER

@y5nw
Copy link
Contributor

y5nw commented Feb 26, 2024

Ok, updated. It works as expected for me with LANGUAGE, although I'm unable to test LC_ALL

Using LC_ALL also works.

@rubenwardy rubenwardy force-pushed the change-language-at-runtime branch from 3cc5132 to ab5202c Compare March 3, 2024 15:38
@rubenwardy
Copy link
Contributor Author

Rebased. As the ContentDB dialog now requests localised meta, I've added a reset_contentdb() function to clear the packages when changing the language

@rubenwardy
Copy link
Contributor Author

rubenwardy commented Mar 3, 2024

@wsor4035 tested on Windows using the mingw and MSVC builds from GitHub actions. The Mingw build passes the test plan in the opening post, including detecting system language. The MSVC build is not compiled with gettext so does not support translation

MSVC left, mingw right

image

@y5nw
Copy link
Contributor

y5nw commented Mar 3, 2024

MSVC appears to be particularly hard to work with (at least from what I tried in a VM).

It seems like gettext[tools] is needed to build Minetest with MSVC and Gettext. I was able to build this with the following patch:

diff --git a/src/gettext.cpp b/src/gettext.cpp
index 972519116..8825d518d 100644
--- a/src/gettext.cpp
+++ b/src/gettext.cpp
@@ -34,6 +34,9 @@ with this program; if not, write to the Free Software Foundation, Inc.,
 #define setlocale(category, localename) \
 	setlocale(category, MSVC_LocaleLookup(localename))
 
+static int argc;
+static char** argv;
+
 static std::map<std::wstring, std::wstring> glb_supported_locales;
 
 /******************************************************************************/
@@ -115,7 +118,7 @@ static const char* MSVC_LocaleLookup(const char* raw_shortname)
 	return "";
 }
 
-static void MSVC_LocaleWorkaround(int argc, char* argv[])
+static void MSVC_LocaleWorkaround()
 {
 	errorstream << "MSVC localization workaround active.  "
 		"Restarting " PROJECT_NAME_C " in a new environment!" << std::endl;
@@ -183,7 +186,7 @@ namespace {
 
 /******************************************************************************/
 void init_gettext(const char *path, const std::string &configured_language,
-	int argc, char *argv[])
+	int _argc, char *_argv[])
 {
 #if USE_GETTEXT
 	char *system_language_c = getenv("LANGUAGE");
@@ -205,6 +208,10 @@ void init_gettext(const char *path, const std::string &configured_language,
 	assert(tdomain);
 	if (tdomain)
 		bind_textdomain_codeset(tdomain, "UTF-8");
+#ifdef _MSC_VER
+	argc = _argc;
+	argv = _argv;
+#endif
 #endif
 
 #else
@@ -254,21 +261,23 @@ void set_gettext_language(std::string configured_language)
 #ifndef SERVER
 	// Hack to force gettext to see the right environment
 	if (current_language != configured_language)
-		MSVC_LocaleWorkaround(argc, argv);
+		MSVC_LocaleWorkaround();
 #else
 	errorstream << "*******************************************************" << std::endl;
 	errorstream << "Can't apply locale workaround for server!" << std::endl;
 	errorstream << "Expect language to be broken!" << std::endl;
 	errorstream << "*******************************************************" << std::endl;
 #endif
-#endif
+#else
 
 	// Notify gettext that the language has changed
 	// Source: https://www.gnu.org/software/gettext/manual/html_node/gettext-grok.html
+	// This does not seem to work when building with MSVC
 	{
 		extern int  _nl_msg_cat_cntr;
 		++_nl_msg_cat_cntr;
 	}
+#endif
 #endif
 
 	/* no matter what locale is used we need number format to be "C" */

However, switching the language causes MT to crash. Edit: it does also spawn a new instance of MT with the selected language. However, the language setting is not changed.

2024-03-03-22-03-58

Stack trace reported by Visual Studio 2022

 	ucrtbased.dll!00007fffc81b2fb5()	未知
 	ucrtbased.dll!00007fffc81b3153()	未知
 	ucrtbased.dll!00007fffc81cae2d()	未知
 	msvcp140d.dll!00007fffcbd426b1()	未知
 	msvcp140d.dll!00007fffcbd423e2()	未知
>	minetest.exe!std::_Mutex_base::~_Mutex_base() 行 50	C++
 	minetest.exe!std::mutex::~mutex()	C++
 	minetest.exe!Settings::~Settings() 行 131	C++
 	[外部代码]	
 	minetest.exe!uninit_common() 行 726	C++
 	[外部代码]	
 	minetest.exe!MSVC_LocaleWorkaround() 行 150	C++
 	minetest.exe!set_gettext_language(std::string configured_language) 行 285	C++
 	minetest.exe!<lambda_69150f7968cdfca1ce97613eb7c6d8d4>::operator()<std::string,void *>(std::string _name, void * _data) 行 712	C++
 	minetest.exe!<lambda_69150f7968cdfca1ce97613eb7c6d8d4>::<lambda_invoker_cdecl><std::string const &,void *>(const std::string & _name, void * _data) 行 713	C++
 	minetest.exe!Settings::doCallbacks(const std::string & name) 行 1092	C++
 	minetest.exe!Settings::set(const std::string & name, const std::string & value) 行 871	C++
 	minetest.exe!LuaSettings::l_set(lua_State * L) 行 205	C++
 	minetest.exe!script_exception_wrapper(lua_State * L, int(*)(lua_State *) f) 行 41	C++
 	[外部代码]	
 	lua51.dll!lua_pcall(lua_State * L, int nargs, int nresults, int errfunc) 行 1145	C
 	minetest.exe!ScriptApiMainMenu::handleMainMenuButtons(const std::unordered_map<std::string,std::string,std::hash<std::string>,std::equal_to<std::string>,std::allocator<std::pair<std::string const ,std::string>>> & fields) 行 91	C++
 	minetest.exe!TextDestGuiEngine::gotText(const std::unordered_map<std::string,std::string,std::hash<std::string>,std::equal_to<std::string>,std::allocator<std::pair<std::string const ,std::string>>> & fields) 行 53	C++
 	minetest.exe!GUIFormSpecMenu::acceptInput(FormspecQuitMode quitmode) 行 4068	C++
 	minetest.exe!GUIFormSpecMenu::OnEvent(const irr::SEvent & event) 行 4999	C++
 	minetest.exe!irr::gui::IGUIElement::OnEvent(const irr::SEvent & event) 行 549	C++
 	minetest.exe!irr::gui::IGUIElement::OnEvent(const irr::SEvent & event) 行 549	C++
 	minetest.exe!GUIScrollContainer::OnEvent(const irr::SEvent & event) 行 57	C++
 	minetest.exe!irr::gui::CGUIComboBox::sendSelectionChangedEvent() 行 377	C++
 	minetest.exe!irr::gui::CGUIComboBox::OnEvent(const irr::SEvent & event) 行 298	C++
 	minetest.exe!irr::gui::CGUIListBox::selectNew(int ypos, bool onlyHover) 行 475	C++
 	minetest.exe!irr::gui::CGUIListBox::OnEvent(const irr::SEvent & event) 行 426	C++
 	minetest.exe!irr::gui::CGUIEnvironment::postEventFromUser(const irr::SEvent & event) 行 597	C++
 	minetest.exe!irr::CIrrDeviceStub::postEventFromUser(const irr::SEvent & event) 行 224	C++
 	minetest.exe!irr::CIrrDeviceSDL::run() 行 695	C++
 	minetest.exe!RenderingEngine::run() 行 149	C++
 	minetest.exe!GUIEngine::run() 行 329	C++
 	minetest.exe!GUIEngine::GUIEngine(JoystickController * joystick, irr::gui::IGUIElement * parent, RenderingEngine * rendering_engine, IMenuManager * menumgr, MainMenuData * data, bool & kill) 行 205	C++
 	minetest.exe!ClientLauncher::main_menu(MainMenuData * menudata) 行 551	C++
 	minetest.exe!ClientLauncher::launch_game(std::string & error_message, bool reconnect_requested, GameStartData & start_data, const Settings & cmd_args) 行 427	C++
 	minetest.exe!ClientLauncher::run(GameStartData & start_data, const Settings & cmd_args) 行 191	C++
 	minetest.exe!main(int argc, char * * argv) 行 265	C++
 	[外部代码]	

@rubenwardy
Copy link
Contributor Author

rubenwardy commented Mar 3, 2024

Yeah, I didn't read MSVC_LocaleWorkaround - restarting the app like that is incompatible with the changing at runtime thing.

We should probably see if it's still needed - this workaround was added 11 years ago ( 22a59b3 ) and so might not be needed any more. I suggest commenting out MSVC_LocaleWorkaround and seeing if it works

@y5nw
Copy link
Contributor

y5nw commented Mar 3, 2024

Does it work on master?

MSVC+Gettext seems to work on the master branch, but without language switching (obviously).

We should probably see if it's still needed - this workaround was added 11 years ago ( 22a59b3 ) and so might not be needed any more. I suggest commenting out MSVC_LocaleWorkaround and seeing if it works

I tried removing the workaround. The result is that changing the language only updates the translations for strings where a translation is previously not available:

2024-03-03-22-17-20
2024-03-03-22-17-25

That said, I removed the _nl_msg_cat_cntr++ code when compiling with MSVC as it resulted in an error, so I am not sure how relevant that is. Edit: I checked again. With the two lines the linker complains that _nl_msg_cat_cntr cannot be found.

IMO it would be best here to change set_gettext_language to no-op for MSVC for now unless someone comes up with a proper solution that does not involve something like this workaround.

@rubenwardy rubenwardy added the Action / change needed Code still needs changes (PR) / more information requested (Issues) label Mar 4, 2024
@@ -232,14 +208,73 @@ void init_gettext(const char *path, const std::string &configured_language,
#endif
Copy link
Contributor

@y5nw y5nw Mar 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO the _WIN32 check here should be removed since some locales on Linux also do not use a UTF-8 locale by default.

@Zughy Zughy added the Adoption needed The pull request needs someone to adopt it. Adoption welcomed! label Jun 1, 2024
@Zughy Zughy closed this Jun 1, 2024
@rubenwardy rubenwardy removed the Adoption needed The pull request needs someone to adopt it. Adoption welcomed! label Jun 3, 2024
@rubenwardy rubenwardy added the Adoption needed The pull request needs someone to adopt it. Adoption welcomed! label Jun 3, 2024
@y5nw y5nw mentioned this pull request Jul 13, 2024
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Action / change needed Code still needs changes (PR) / more information requested (Issues) Adoption needed The pull request needs someone to adopt it. Adoption welcomed! @ Mainmenu @ Translation UI/UX
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants