Commit Graph
50 Commits
Author SHA1 Message Date
Aussiemon 74b6d284b0 Fix read issue with ESM4 compressed data 2026-10-25 23:08:22 -06:00
Evil Eye 6e8b6ef48e Don't use filesystem::path while looking for vfs files 2025-12-08 16:06:15 +01:00
elsid 4186ca6ceb Fix clang-analyzer-security.ArrayBound warning
Size of an array is not a valid index.

components/esm4/reader.cpp:925:15: error: Out of bound access to memory after the end of 'sGroupType' [clang-analyzer-security.ArrayBound,-warnings-as-errors]
  925 |         ss << sGroupType[std::min<std::size_t>(type, std::size(sGroupType))]; // avoid out of range
      |               ^
components/esm4/reader.cpp:627:13: note: Assuming field 'groupSize' is equal to field 'recHeaderSize'
  627 |         if (mCtx.recordHeader.group.groupSize == (std::uint32_t)mCtx.recHeaderSize)
      |             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
components/esm4/reader.cpp:627:9: note: Taking true branch
  627 |         if (mCtx.recordHeader.group.groupSize == (std::uint32_t)mCtx.recHeaderSize)
      |         ^
components/esm4/reader.cpp:634:17: note: Assuming the condition is true
  634 |             if (!mCtx.groupStack.empty()) // top group may be empty (e.g. HAIR in Skyrim)
      |                 ^~~~~~~~~~~~~~~~~~~~~~~~
components/esm4/reader.cpp:634:13: note: Taking true branch
  634 |             if (!mCtx.groupStack.empty()) // top group may be empty (e.g. HAIR in Skyrim)
      |             ^
components/esm4/reader.cpp:638:17: note: Calling 'Reader::exitGroupCheck'
  638 |                 exitGroupCheck();
      |                 ^~~~~~~~~~~~~~~~
components/esm4/reader.cpp:650:13: note: Assuming the condition is false
  650 |         if (mCtx.groupStack.empty())
      |             ^~~~~~~~~~~~~~~~~~~~~~~
components/esm4/reader.cpp:650:9: note: Taking false branch
  650 |         if (mCtx.groupStack.empty())
      |         ^
components/esm4/reader.cpp:655:16: note: Assuming 'lastGroupSize' is <= field 'second'
  655 |         while (mCtx.groupStack.back().second >= lastGroupSize)
      |                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
components/esm4/reader.cpp:655:9: note: Loop condition is true.  Entering loop body
  655 |         while (mCtx.groupStack.back().second >= lastGroupSize)
      |         ^
components/esm4/reader.cpp:663:17: note: Assuming 'overshoot' is <= 0
  663 |             if (overshoot > 0)
      |                 ^~~~~~~~~~~~~
components/esm4/reader.cpp:663:13: note: Taking false branch
  663 |             if (overshoot > 0)
      |             ^
components/esm4/reader.cpp:676:17: note: Assuming the condition is false
  676 |             if (mCtx.groupStack.empty())
      |                 ^~~~~~~~~~~~~~~~~~~~~~~
components/esm4/reader.cpp:676:13: note: Taking false branch
  676 |             if (mCtx.groupStack.empty())
      |             ^
components/esm4/reader.cpp:682:17: note: Assuming 'lastGroupSize' is >= field 'second'
  682 |             if (lastGroupSize < mCtx.groupStack.back().second)
      |                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
components/esm4/reader.cpp:682:13: note: Taking false branch
  682 |             if (lastGroupSize < mCtx.groupStack.back().second)
      |             ^
components/esm4/reader.cpp:686:17: note: Assuming 'lastGroupSize' is < field 'second'
  686 |             if (mCtx.groupStack.back().second > lastGroupSize) // FIXME: debugging only
      |                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
components/esm4/reader.cpp:686:13: note: Taking true branch
  686 |             if (mCtx.groupStack.back().second > lastGroupSize) // FIXME: debugging only
      |             ^
components/esm4/reader.cpp:687:30: note: Calling 'printLabel'
  687 |                 std::cerr << printLabel(mCtx.groupStack.back().first.label, mCtx.groupStack.back().first.type)
      |                              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
components/esm4/reader.cpp:925:15: note: Access of 'sGroupType' at index 12, while it holds only 12 'class std::basic_string_view<char>' elements
  925 |         ss << sGroupType[std::min<std::size_t>(type, std::size(sGroupType))]; // avoid out of range
      |               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2025-11-25 00:03:39 +01:00
Alexei Kotov ac9505b536 Rename components/to_utf8 directory and files to follow naming conventions 2025-02-23 00:18:07 +03:00
elsid 2ddc77138a Use normalized path in ESM4 reader 2024-09-18 23:52:20 +02:00
AnyOldName3 da8150e2e4 Even more MSVC-specific warnings that evaded detection in CI 2024-03-21 15:51:29 +00:00
Alexei Kotov 6b93479bd3 Get rid of ESM4::SubRecordTypes
All my homies hate ESM4::SubRecordTypes
2024-03-18 12:25:17 +03:00
elsid 199d97d32a Use forward declaration for VFS::Manager 2024-01-24 21:52:48 +01:00
Alexei Kotov 8c27dca1df ESM4: add a way to get the current form version
Differentiate between Fallout 4 and TES4 version 1.0 plugins
2023-09-25 21:26:51 +03:00
Andrei Kortunov 5d51f3ea26 Init structs to make sure that we do not store a random data 2023-09-04 12:19:56 +04:00
Alexei Kotov b7efdc4add Use the correct filenames for FO4 string tables 2023-08-17 02:14:22 +03:00
Alexei Kotov 47af96eb12 Generalize extended storage subrecord handling 2023-08-17 02:14:22 +03:00
Petr Mikheev 5d54ca6fdb Refactor components/esm4 2023-08-05 13:12:24 +02:00
Andrei Kortunov 1ae2a994f0 Avoid null pointer dereference 2023-07-31 16:35:29 +04:00
elsid 11204d35d6 Use zlib directly for ESM4
Fallback to decompression by block of fixed size when decompressing the
whole archive fails.
2023-06-10 00:12:57 +02:00
jvoisin 3152078c7e Restore missing std::endl removed by an overeager regex 2023-06-01 13:48:41 +02:00
jvoisin 0e33730f5d Remove useless logging
As suggested in https://gitlab.com/OpenMW/openmw/-/merge_requests/3086#note_1413234840
2023-06-01 13:31:48 +02:00
jvoisin a58dc6fd30 Use the logging system instead of std::cout in components/esm4
- Remove debug-related includes
- Add some trivial missing includes
- Remove useless {}
- Move the known-but-skipped-records are the end of the switch-cases
- Always throw on unknown records
2023-06-01 13:31:48 +02:00
elsid e537d1d0d4 Ignore missing localization string files by esmtool
Dawnguard.esm from Skyrim depends on files stored in Dawnguard.bsa which is not
processed by esmtool.
2023-05-22 22:09:08 +02:00
elsid 358b7ad3ec Keep ESM4 localized strings in memory
Size of the files is in order of megabytes at max. Storing offset lookup table
to read from file on demand is less efficient than reading from memory for such
size.

Read and store offsets first. Sort them to read values sequentially. Memoize
last offset and value to avoid reading the same value twice. Use seek only when
current possition does not match offset. Optimize seek for short distance by
calling read instead.
2023-05-22 22:09:06 +02:00
Alexei Kotov 401ebfd7f2 Merge branch 'fix_oblivion_loading' into 'master'
Fix Oblivion loading

See merge request OpenMW/openmw!3050
2023-05-22 18:18:35 +00:00
jvoisin a9bf671992 Fix Oblivion loading
Found by elsid@, and due to !3043
2023-05-22 18:24:39 +02:00
jvoisin 8b654f276d Replace all the asserts with throw in components/esm4/reader.cpp 2023-05-22 17:42:32 +02:00
jvoisin 534e20cf3a Replace another assert with a throw
It fixes the following crash:

```nasm
[----------------------------------registers-----------------------------------]
RAX: 0xffffffffffffffe4
RBX: 0x7fffffffbd88 --> 0x0
RCX: 0x555555861f01 --> 0xd6000055555582fa
RDX: 0x1
RSI: 0x0
RDI: 0xff
RBP: 0x2e94
RSP: 0x7fffffff9bb0 --> 0x0
RIP: 0x55555574e286 (<_ZN4ESM44Cell4loadERNS_6ReaderE+182>:	mov    eax,DWORD PTR [rax+0xc])
R8 : 0x0
R9 : 0x555555836bd0 --> 0x555555861f90 --> 0x100000000000000
R10: 0xffffffff
R11: 0x0
R12: 0x7fffffffbd88 --> 0x0
R13: 0x7fffffffbd30 --> 0x7fffffffbd28 --> 0xff0001
R14: 0x7fffffff9da0 --> 0x10fc
R15: 0x7fffffff9ea8 --> 0x7fffffff9eb8 --> 0x0
EFLAGS: 0x10206 (carry PARITY adjust zero sign trap INTERRUPT direction overflow)
[-------------------------------------code-------------------------------------]
   0x55555574e27c <_ZN4ESM44Cell4loadERNS_6ReaderE+172>:	mov    rdi,r12
   0x55555574e27f <_ZN4ESM44Cell4loadERNS_6ReaderE+175>:	xor    esi,esi
   0x55555574e281 <_ZN4ESM44Cell4loadERNS_6ReaderE+177>:	call   0x5555557edfb0 <_ZNK4ESM46Reader3grpEm>
=> 0x55555574e286 <_ZN4ESM44Cell4loadERNS_6ReaderE+182>:	mov    eax,DWORD PTR [rax+0xc]
   0x55555574e289 <_ZN4ESM44Cell4loadERNS_6ReaderE+185>:	cmp    eax,0xff
   0x55555574e28e <_ZN4ESM44Cell4loadERNS_6ReaderE+190>:	ja     0x55555574e314 <_ZN4ESM44Cell4loadERNS_6ReaderE+324>
   0x55555574e294 <_ZN4ESM44Cell4loadERNS_6ReaderE+196>:	cmp    al,0x5
   0x55555574e296 <_ZN4ESM44Cell4loadERNS_6ReaderE+198>:	jne    0x55555574e314 <_ZN4ESM44Cell4loadERNS_6ReaderE+324>
[------------------------------------stack-------------------------------------]
0000| 0x7fffffff9bb0 --> 0x0
0008| 0x7fffffff9bb8 --> 0x0
0016| 0x7fffffff9bc0 --> 0x0
0024| 0x7fffffff9bc8 --> 0x0
0032| 0x7fffffff9bd0 --> 0x0
0040| 0x7fffffff9bd8 --> 0x0
0048| 0x7fffffff9be0 --> 0x0
0056| 0x7fffffff9be8 --> 0x0
[------------------------------------------------------------------------------]
Legend: code, data, rodata, value
Stopped reason: SIGSEGV
0x000055555574e286 in ESM4::Cell::load (this=0x7fffffff9da0, reader=...) at /home/jvoisin/dev/openmw/openmw/components/esm4/loadcell.cpp:68
68	   if (reader.grp().type == ESM4::Grp_ExteriorSubCell && reader.grp().label.grid[1] == 0
gdb-peda$ bt
    this=<optimized out>, reader=...) at /home/jvoisin/dev/openmw/openmw/apps/esmtool/tes4.cpp:547
    recordInvocable=...) at /home/jvoisin/dev/openmw/openmw/./components/esm4/readerutils.hpp:31
    at /home/jvoisin/dev/openmw/openmw/./components/esm4/readerutils.hpp:23
    at ../csu/libc-start.c:381
```
2023-05-21 20:07:52 +02:00
jvoisin cde7aeac3d Replace an asert with a conditional throw
The assert is reachable, resulting in a brutal `abort`
instead of a nice exception.
2023-05-20 19:33:14 +02:00
Bret Curtis 95fff7b77b show which file is not found 2023-05-10 21:43:55 +02:00
Petr Mikheev ff774d6406 Load more ESM4 records 2023-04-09 18:52:11 +02:00
Petr Mikheev f09a689a4f Merge ESM::RefNum and ESM4::FormId 2023-04-09 00:23:32 +02:00
elsid dbed00842e Split components/esm4/common.hpp header 2023-04-04 10:12:20 +02:00
florent.teppe a3a7767093 applies review changes
filepos changed to an actual file pos
moved lambda declaration out of function call
2023-03-28 17:11:30 +02:00
florent.teppe 464092e323 fix oblivion and skyrim 2023-03-28 17:11:30 +02:00
florent.teppe 34dd24b261 Initial changes to detect when context isn't usable 2023-03-28 17:11:30 +02:00
elsid 67e96d48ba Use relative to content file path to find strings file
To be used by esmtool when VFS is not used.
2023-03-18 03:00:18 +01:00
Cédric Mocquillon 6e23ad82d5 Load esm files from vfs 2023-02-19 21:58:19 +01:00
clang-format-botandζeh Matt ddb0522bbf Apply clang-format to code base 2022-09-22 21:35:26 +03:00
elsid 6f00641c8a Fix reading array of zero terminated strings
NIFZ and KFFZ subrecords store multiple strings separated by \0 character.
2022-09-15 00:01:19 +02:00
elsid 4f7dcd5ae5 Fix conversion to UTF8 for ESM4 Reader strings
toUtf8 returns a string view to the input when input string is ASCII and nothing
is written to the buffer which means output string of Reader::getStringImpl is
not modified.

Move input to the output string and resize it in this case.
2022-09-12 01:42:01 +02:00
Project579 886b8c7af2 Fix build error with stdlibc++ due to includes shuffling. 2022-09-11 14:41:21 +02:00
Project579 a13709c510 Replace implicit convertions from std::filesystem::path to std::string with correctly converting functions. 2022-09-11 14:41:20 +02:00
Project579 e5c417c968 Make sure all paths are passed as std::filesystem::path instead of std::string where possible. 2022-09-11 14:41:15 +02:00
elsid ac1688f9c1 Remove ESM::Reader base class
This class does not serve any useful purpose now. It was added as a first step
in attempt to define common interface for ESM3 and ESM4 readers. But this is not
going to happen.
2022-09-10 15:17:49 +02:00
elsid f99ed6f1db Split components/misc/stringops.hpp into multiple headers
Replace all ciEqual overloads with one having std::string_view as argument.
2022-08-03 22:06:24 +02:00
elsid 6c8ed4d19c Move Files::IStreamPtr alias to a separate header
To avoid transitive include of Windows.h all over the engine.
2022-07-17 11:35:39 +02:00
Andrei Kortunov 14b5674d32 Init missing data members 2022-07-03 15:44:50 +04:00
jvoisin aeb3733e57 Remove boost:: constructs
Namely boost::scoped_array and boost::to_lower_copy.
2022-05-29 21:21:51 +02:00
jvoisinandPetr Mikheev 4e4debb1cb Remove even more of boost::filesystem 2022-05-25 18:29:02 +00:00
Evil Eye a64979e25d Replace empty std::string assignments 2022-05-04 22:33:39 +02:00
elsid 94c1d0cced Use unique_ptr to store istream 2022-04-22 18:27:17 +02:00
elsid 5eb8c4aebe Avoid redundant conversion to const char* and use make_shared 2022-04-10 22:33:08 +02:00
cc9ciiandBret Curtis ba3ae448d4 Initial import of esm4 by cc9cii 2022-01-31 08:58:45 +01:00