Skip to content

Commit

Permalink
[Very Important Module Updates And Fixes (非常重要的模块更新和修复)]
Browse files Browse the repository at this point in the history 
Please do not delete this commit! (请不要删除此提交!)

Tips: EODF in-development 0.2.0 version will be released soon
提示: 即将发布 EODF in-development 0.2.0 版本

THE CURRENT PROJECT IS ROLLING AND THE VERSION IS PREVIEW IN-DEVELOPMENT 0.2.0
目前的项目是滚动的，版本是开发中的预览版本 0.2.0

English:

1.Experimental version with AES module removed
2.Refactor TripleDES module to the official version and keep the experimental version
3."Deterministic random number generation" using HMAC algorithm is used to construct polynomials to support Shamir's secret sharing.
4.Secret sharing algorithm of Shamir‘s with Galois finite field (2^8)
5.Collate the original random number generation engine and append a utility class for secure random number seeding
6.Give the project uniform use, MSVC's library of random number distribution functions, collation and duplication (15%)
7.Update UnitTester.hpp

Chinese:

1.删除AES模块的实验性版本
2.将TripleDES模块重构为正式版本，保留实验版本
3.使用HMAC算法的 "确定性随机数生成 "来构建多项式，以支持Shamir的秘密共享。
4.用伽罗瓦有限域（2^8）的Shamir's秘密共享算法。
5.整理原随机数生成引擎，并附加一个用于安全随机数播种的实用类
6.给项目统一使用，MSVC的随机数分布函数库，整理和复制（15%）。
7.更新UnitTester.hpp

/*************************************************************************************************************************************/

English:

About reference commit (21f1af4) message:
The memory fill function is a very important basic function.
The previous memory fill function, which was a serious violation of data security regulations, must now be removed! Because the previous memory fill function had certain compiler "optimizations" (the address where the fill function was called was not generated in the assembly code), possible changes in the fill data were not written to the memory address.
This time, memory fill functions are prohibited from compiler optimizations due to their volatile nature! Two types of functions are now provided: (functions where the safe fill data is zero, and functions where the safe fill data is otherwise the same data)

Fixes:

1. Fix bugs and correct incorrect word writing,

2. Performance optimization AES algorithm ( finished ) TripleDES algorithm ( finished )

3. Pre-reconfigured source code content

4. Trying to fix the Blake3 hash hashing algorithm (progress: 100%)

5. Fixed the stream cryptograph handler function and the call to the memory data fill function, which found the wrong parameters and usage

Security Update:
1.[DO FIXED] Clean up temporary sensitive data as necessary in order to prevent memory data from being illegally Dumped, thus affecting user data security

2.Fix Xorshift256 PRNG and Make Xorshift1024 PRNG

3.A reconstruction of Shannon's proposed, pseudo-random number generator made based on the method of taking the middle part of the square. and its improved version of the paper CBPRNG.

Makes:
1. Feedback shift register (Linear and Nonlinear version)
The non-linear version was designed by Twilight-Dream and passed 2531 tests with the PractRand evaluation program

2. A symmetric encryption and decryption algorithm from China Business (ShangYongMiMa) Cryptography 4 (ECB Mode)
https://datatracker.ietf.org/doc/html/draft-ribose-cfrg-sm4

3. Twofish encryption and decryption algorithm has been completed
https://en.wikipedia.org/wiki/Twofish

4. Cryptography The padding data function does modular processing

5. A chaos-theoretic system based on the simulation of physical two-segment pendulum motion phenomena, involving a random number generator for differential equations

Chinese:

关于参考提交
（21f1af4）
信息:
内存填充功能是一个非常重要的基本功能。
以前的内存填充功能，严重违反了数据安全规定，现在必须删除! 因为以前的内存填充函数有一定的编译器 "优化"（调用填充函数的地址没有在汇编代码中生成），填充数据的可能变化没有被写入内存地址。
这一次，由于内存填充函数的易失性，禁止对其进行编译器优化! 现在提供了两种类型的函数：（安全填充数据为零的函数，以及安全填充数据为其他相同数据的函数）

修复:
1. 修复错误和纠正不正确的词汇书写

2. 性能优化 AES算法(完成的) TripleDES算法(完成的)

3. 预重构源代码内容

4. 尝试修复Blake3哈希散列算法(进度: 100%)

5. 修复了流密码器的处理函数，以及调用内存数据填充函数，发现了错误的参数和使用方法

安全更新:
1. [DO FIXED] 必要时清理临时敏感数据，以防止内存数据被非法Dump，从而影响用户数据安全

2. 修复Xorshift256 PRNG和制作Xorshift1024 PRNG

3. 重构了香农提出的，基于平方取中间部分的方法，做成的伪随机数生成器。以及它的论文改进版本CBPRNG。

制作:
1. 反馈移位寄存器（线性和非线性版本）
非线性版本由Twilight-Dream设计，通过了PractRand评估程序的2531项测试

2. 中国商用(ShangYongMiMa)密码4(ECB模式)的一种对称加解密算法
https://datatracker.ietf.org/doc/html/draft-ribose-cfrg-sm4

3. Twofish加解密算法已经完成
https://en.wikipedia.org/wiki/Twofish

4. 密码学填充数据功能做模块化处理

5. 属于混沌理论系统，基于物理双段摆锤运动现象的模拟，涉及微分方程的随机数生成器。
  • Loading branch information
@Twilight-Dream-Of-Magic
Twilight-Dream-Of-Magic committed Nov 8, 2022
1 parent fc32f21 commit ababf43
Show file tree
Hide file tree
Showing 39 changed files with 22,313 additions and 11,546 deletions.
7,182 changes: 4,317 additions & 2,865 deletions include/CommonSecurity/BlockDataCryption.hpp
View file

Large diffs are not rendered by default.

4,000 changes: 1,191 additions & 2,809 deletions include/CommonSecurity/CommonSecurity.hpp
View file

Large diffs are not rendered by default.

92 changes: 52 additions & 40 deletions include/CommonSecurity/DataHashingWrapper.hpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -218,7 +218,7 @@ namespace CommonSecurity::DataHashingWrapper
(
std::size_t RandomNumberSeed,
std::size_t RandomNumberSeed2,
std::vector<unsigned char>& ProcessData,
std::vector<std::uint8_t>& ProcessData,
bool IsEncodeOrDecodeMode
)
{
Expand Down Expand Up @@ -268,11 +268,11 @@ namespace CommonSecurity::DataHashingWrapper
//Original password (Conactenate operation)
std::string CombinedMultiPasswordString = MultiPasswordString[ 0 ] + MultiPasswordString[ 1 ] + MultiPasswordString[ 2 ] + MultiPasswordString[ 3 ];

std::vector<MySupport_Library::Types::my_ulli_type> PasswordStringIntegers;
std::vector<std::uint64_t> PasswordStringIntegers;

for(auto& PasswordString : MultiPasswordString)
{
auto TemporaryPasswordStringIntegers = StringToInteger<MySupport_Library::Types::my_ulli_type>( PasswordString );
auto TemporaryPasswordStringIntegers = StringToInteger<std::uint64_t>( PasswordString );
PasswordStringIntegers.insert(PasswordStringIntegers.end(), TemporaryPasswordStringIntegers.begin(), TemporaryPasswordStringIntegers.end());

TemporaryPasswordStringIntegers.clear();
Expand Down Expand Up @@ -351,16 +351,16 @@ namespace CommonSecurity::DataHashingWrapper

#else

std::vector<unsigned char> ExtendedChacha20_Message = ASCII_Hexadecmial::hexadecimalString2ByteArray(HashMessage);
std::vector<std::uint8_t> ExtendedChacha20_Message = ASCII_Hexadecmial::hexadecimalString2ByteArray(HashMessage);

std::deque<std::vector<unsigned char>> ExtendedChacha20_Nonces;
std::deque<std::vector<std::uint8_t>> ExtendedChacha20_Nonces;

for( auto& PasswordHashedString : MultiPasswordHashedString )
{
ExtendedChacha20_Nonces.push_back( ASCII_Hexadecmial::hexadecimalString2ByteArray(PasswordHashedString) );
}

std::vector<unsigned char> ExtendedChacha20_Key;
std::vector<std::uint8_t> ExtendedChacha20_Key;

//Does it use a true random number generator?
//是否使用真随机数生成器？
Expand All @@ -382,6 +382,9 @@ namespace CommonSecurity::DataHashingWrapper

auto ExportedObfuscatorResultTable = this->ApplyCustomDataObfuscation<false>(CurrentRandomSeed, CurrentRandomSeed2, ExtendedChacha20_Key, true);

CurrentRandomSeed = 0;
CurrentRandomSeed2 = 0;

std::destroy_at(&ExportedObfuscatorResultTable);
}
else
Expand All @@ -403,20 +406,23 @@ namespace CommonSecurity::DataHashingWrapper

auto ExportedObfuscatorResultTable = this->ApplyCustomDataObfuscation<false>(X_Seed, Y_Seed, ExtendedChacha20_Key, true);

X_Seed = 0;
Y_Seed = 0;

std::destroy_at(&ExportedObfuscatorResultTable);
}

std::vector<unsigned char> ExtendedChacha20_InitialKey(32, 0xFF);
std::vector<std::uint8_t> ExtendedChacha20_InitialKey(32, 0xFF);
CommonSecurity::StreamDataCryptographic::ExtendedChaCha20 ExtendedChacha20_IETF(ExtendedChacha20_InitialKey);

//Apply ExtendedChacha20-IETF
//应用ExtendedChacha20-IETF

std::vector<unsigned char> ThisProcessedMessage;
std::vector<std::uint8_t> ThisProcessedMessage;
for( auto& ExtendedChacha20_Nonce : ExtendedChacha20_Nonces )
{
std::vector<unsigned char> ExtendedChacha20_UsingKey = ExtendedChacha20_Key;
std::vector<unsigned char> ThisProcessedMessage = CommonSecurity::StreamDataCryptographic::Helpers::Helper(ExtendedChacha20_IETF, ExtendedChacha20_Message, ExtendedChacha20_UsingKey, ExtendedChacha20_Nonce);
std::vector<std::uint8_t> ExtendedChacha20_UsingKey = ExtendedChacha20_Key;
std::vector<std::uint8_t> ThisProcessedMessage = CommonSecurity::StreamDataCryptographic::Helpers::Helper(ExtendedChacha20_IETF, ExtendedChacha20_Message, ExtendedChacha20_UsingKey, ExtendedChacha20_Nonce);
HashMessageStringOfKeys.push_back( ASCII_Hexadecmial::byteArray2HexadecimalString(ThisProcessedMessage) );

if(ExtendedChacha20_Message.empty())
Expand Down Expand Up @@ -481,11 +487,11 @@ namespace CommonSecurity::DataHashingWrapper
if constexpr(false)
{
std::vector<std::uint32_t> RandomNumberSeedSequence = GenerateSecureRandomNumberSeedSequence<std::uint32_t>(PasswordStreamWords.size());
PRNE.InitialBySeed( RandomNumberSeedSequence.begin(), RandomNumberSeedSequence.end(), 0, false );
PRNE.InitialBySeed<std::uint32_t, std::vector<std::uint32_t>::iterator>( RandomNumberSeedSequence.begin(), RandomNumberSeedSequence.end(), false );
}
else
{
PRNE.InitialBySeed( PasswordStreamWords.begin(), PasswordStreamWords.end(), 0, false );
PRNE.InitialBySeed<std::uint32_t, std::vector<std::uint32_t>::iterator>( PasswordStreamWords.begin(), PasswordStreamWords.end(), false );
}

CommonSecurity::ShuffleRangeData( PasswordStreamWords.begin(), PasswordStreamWords.end(), PRNE.random_generator );
Expand All @@ -498,6 +504,16 @@ namespace CommonSecurity::DataHashingWrapper
CommonSecurity::ShuffleRangeData( PasswordStreamWords.begin(), PasswordStreamWords.end(), PRNE.random_generator );

PasswordStreamBytes = CommonToolkit::MessageUnpacking<std::uint32_t, std::uint8_t>( PasswordStreamWords.data(), PasswordStreamWords.size() );

volatile void* CheckPointer = nullptr;

CheckPointer = memory_set_no_optimize_function<0x00>(PasswordStreamWords.data(), sizeof(std::uint32_t) * PasswordStreamWords.size());
if(CheckPointer != PasswordStreamWords.data())
{
throw std::runtime_error("Force Memory Fill Has Been \"Optimization\" !");
}


PasswordStreamWords.clear();
PasswordStreamWords.shrink_to_fit();
}
Expand All @@ -522,9 +538,15 @@ namespace CommonSecurity::DataHashingWrapper

~HashTokenForData()
{
for( auto& StringData: OriginalPasswordStrings )
volatile void* CheckPointer = nullptr;

for( auto& StringData : OriginalPasswordStrings )
{
memory_set_no_optimize_function(std::addressof(StringData), 0, StringData.size());
memory_set_no_optimize_function<0x00>(StringData.data(), StringData.size());
if(CheckPointer != StringData.data())
{
throw std::runtime_error("Force Memory Fill Has Been \"Optimization\" !");
}
}

OriginalPasswordStrings.clear();
Expand Down Expand Up @@ -606,7 +628,7 @@ namespace CommonSecurity::DataHashingWrapper

if constexpr (mode == CommonSecurity::SHA::Hasher::WORKER_MODE::ARGON2)
{
using CommonSecurity::KDF::Argon2::Constants::WORDS_BLOCK_SIZE;
using CommonSecurity::KDF::Argon2::Constants::WORDS_MEMORY_BLOCK_SIZE;
using CommonSecurity::KDF::Argon2::Argon2_Parameters;
using CommonSecurity::KDF::Argon2::Argon2;
using CommonSecurity::KDF::Argon2::AlgorithmVersion;
Expand Down Expand Up @@ -636,12 +658,12 @@ namespace CommonSecurity::DataHashingWrapper

std::size_t ThreadNumber = std::thread::hardware_concurrency();

if( GeneratePasswordStreamHashByteTokenSize / (sizeof(std::uint64_t) * WORDS_BLOCK_SIZE) == 0 )
if( GeneratePasswordStreamHashByteTokenSize / (sizeof(std::uint64_t) * WORDS_MEMORY_BLOCK_SIZE) == 0 )
{
GeneratePasswordStreamHashByteTokenSize *= (sizeof(std::uint64_t) * WORDS_BLOCK_SIZE);
GeneratePasswordStreamHashByteTokenSize *= (sizeof(std::uint64_t) * WORDS_MEMORY_BLOCK_SIZE);
}
std::size_t MemoryByteSize = try_allocate_temporary_memory_size(GeneratePasswordStreamHashByteTokenSize * (sizeof(std::uint64_t) * WORDS_BLOCK_SIZE)).value();
std::size_t MemoryBlockSpaceNumber = MemoryByteSize / (sizeof(std::uint64_t) * WORDS_BLOCK_SIZE);
std::size_t MemoryByteSize = try_allocate_temporary_memory_size(GeneratePasswordStreamHashByteTokenSize * (sizeof(std::uint64_t) * WORDS_MEMORY_BLOCK_SIZE)).value();
std::size_t MemoryBlockSpaceNumber = MemoryByteSize / (sizeof(std::uint64_t) * WORDS_MEMORY_BLOCK_SIZE);
std::size_t TimeIterationNumber = MemoryBlockSpaceNumber % 128 == 0 ? MemoryBlockSpaceNumber / 128 : (MemoryBlockSpaceNumber % 128) * 2;

Argon2_Parameters Argon2KDF_ParameterObject
Expand Down Expand Up @@ -721,11 +743,6 @@ namespace CommonSecurity::DataHashingWrapper
PasswordStreamHashedToken_Bytes.shrink_to_fit();
}

for(auto& password : this->OriginalPasswordStrings )
{
memory_set_no_optimize_function(password.data(), 0x00, password.size());
}

return HashKeyStreamTokenResultObject;
}
else
Expand Down Expand Up @@ -753,18 +770,18 @@ namespace CommonSecurity::DataHashingWrapper

//Magic number from std::cout << std::hex << *reinterpret_cast<const unsigned long long *>(&std::numbers::e) << std::endl;
//Magic number from std::cout << std::hex << *reinterpret_cast<const unsigned long long *>(&std::numbers::pi) << std::endl;
constexpr std::array<unsigned int, 4> MagicNumberConstantArray { 0x4005bf0a, 0x8b145769, 0x400921fb, 0x54442d18 };
constexpr std::array<std::uint32_t, 4> MagicNumberConstantArray { 0x4005bf0a, 0x8b145769, 0x400921fb, 0x54442d18 };

//Execute Super TEA "Encrypt" Operation
for(auto& PasswordString : MultiPasswordString)
{
while (PasswordString.size() % sizeof(unsigned int) != 0)
while (PasswordString.size() % sizeof(std::uint32_t) != 0)
PasswordString.push_back(0);

std::vector<unsigned char> classic_bytes{ PasswordString.data(), PasswordString.data() + PasswordString.size() };
std::span<const unsigned char> classic_bytes_span { classic_bytes };
std::vector<unsigned int> word_ascii_codes(classic_bytes_span.size() / 4);
std::span<const unsigned int> word_ascii_codes_span { word_ascii_codes };
std::vector<std::uint8_t> classic_bytes{ PasswordString.data(), PasswordString.data() + PasswordString.size() };
std::span<const std::uint8_t> classic_bytes_span { classic_bytes };
std::vector<std::uint32_t> word_ascii_codes(classic_bytes_span.size() / 4);
std::span<const std::uint32_t> word_ascii_codes_span { word_ascii_codes };

CommonToolkit::MessagePacking(classic_bytes_span, word_ascii_codes.data());

Expand All @@ -790,13 +807,13 @@ namespace CommonSecurity::DataHashingWrapper
//Execute Super TEA "Decrypt" Operation
for(auto& PasswordString : HashedTokenHexadecimalString)
{
while (PasswordString.size() % sizeof(unsigned int) != 0)
while (PasswordString.size() % sizeof(std::uint32_t) != 0)
PasswordString.push_back(0);

std::vector<unsigned char> classic_bytes{ PasswordString.data(), PasswordString.data() + PasswordString.size() };
std::span<const unsigned char> classic_bytes_span { classic_bytes };
std::vector<unsigned int> word_ascii_codes( classic_bytes_span.size() / 4 );
std::span<const unsigned int> word_ascii_codes_span { word_ascii_codes };
std::vector<std::uint8_t> classic_bytes{ PasswordString.data(), PasswordString.data() + PasswordString.size() };
std::span<const std::uint8_t> classic_bytes_span { classic_bytes };
std::vector<std::uint32_t> word_ascii_codes( classic_bytes_span.size() / 4 );
std::span<const std::uint32_t> word_ascii_codes_span { word_ascii_codes };

CommonToolkit::MessagePacking( classic_bytes_span, word_ascii_codes.data() );

Expand Down Expand Up @@ -866,11 +883,6 @@ namespace CommonSecurity::DataHashingWrapper
PasswordStreamHashedToken_Bytes.shrink_to_fit();
}

for(auto& password : this->OriginalPasswordStrings )
{
memory_set_no_optimize_function(password.data(), 0x00, password.size());
}

return HashKeyStreamTokenResultObject;
}
}
Expand Down
Loading

0 comments on commit ababf43

Please sign in to comment.