Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build error: ‘uint32_t’ does not name a type in utf8.h #1

Open
Tejes opened this issue Apr 24, 2024 · 1 comment · May be fixed by #2
Open

Build error: ‘uint32_t’ does not name a type in utf8.h #1

Tejes opened this issue Apr 24, 2024 · 1 comment · May be fixed by #2

Comments

@Tejes
Copy link

Tejes commented Apr 24, 2024

Trying to build this repo (in order to get NeMo running) I get the following error:

$ pip install "git+https://github.com/LahiLuk/YouTokenToMe"
Collecting git+https://github.com/LahiLuk/YouTokenToMe
  Cloning https://github.com/LahiLuk/YouTokenToMe to /tmp/pip-req-build-pbvabuz3
  Running command git clone --filter=blob:none --quiet https://github.com/LahiLuk/YouTokenToMe /tmp/pip-req-build-pbvabuz3
  Resolved https://github.com/LahiLuk/YouTokenToMe to commit f9fe56e198e22d552d821a5f432d4e5ada1e81e8
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting Click>=7.0 (from youtokentome==1.0.6)
  Obtaining dependency information for Click>=7.0 from https://files.pythonhosted.org/packages/00/2e/d53fa4befbf2cfa713304affc7ca780ce4fc1fd8710527771b58311a3229/click-8.1.7-py3-none-any.whl.metadata
  Downloading click-8.1.7-py3-none-any.whl.metadata (3.0 kB)
Using cached click-8.1.7-py3-none-any.whl (97 kB)
Building wheels for collected packages: youtokentome
  Building wheel for youtokentome (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for youtokentome (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [121 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-311
      creating build/lib.linux-x86_64-cpython-311/youtokentome
      copying youtokentome/youtokentome.py -> build/lib.linux-x86_64-cpython-311/youtokentome
      copying youtokentome/__init__.py -> build/lib.linux-x86_64-cpython-311/youtokentome
      copying youtokentome/yttm_cli.py -> build/lib.linux-x86_64-cpython-311/youtokentome
      running build_ext
      building '_youtokentome_cython' extension
      creating build/temp.linux-x86_64-cpython-311
      creating build/temp.linux-x86_64-cpython-311/youtokentome
      creating build/temp.linux-x86_64-cpython-311/youtokentome/cpp
      x86_64-linux-gnu-gcc -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -fstack-clash-protection -Wformat -Werror=format-security -fcf-protection -g -fwrapv -O2 -fPIC -Iyoutokentome/cpp -I/home/tejes/work/inmoment/parakeet/venv/include -I/usr/include/python3.11 -c youtokentome/cpp/bpe.cpp -o build/temp.linux-x86_64-cpython-311/youtokentome/cpp/bpe.o -std=c++11 -pthread -O3
      x86_64-linux-gnu-gcc -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -fstack-clash-protection -Wformat -Werror=format-security -fcf-protection -g -fwrapv -O2 -fPIC -Iyoutokentome/cpp -I/home/tejes/work/inmoment/parakeet/venv/include -I/usr/include/python3.11 -c youtokentome/cpp/utf8.cpp -o build/temp.linux-x86_64-cpython-311/youtokentome/cpp/utf8.o -std=c++11 -pthread -O3
      In file included from youtokentome/cpp/utf8.cpp:1:
      youtokentome/cpp/utf8.h:9:18: error: ‘uint32_t’ does not name a type
          9 | constexpr static uint32_t INVALID_UNICODE = 0x0fffffff;
            |                  ^~~~~~~~
      youtokentome/cpp/utf8.h:6:1: note: ‘uint32_t’ is defined in header ‘<cstdint>’; did you forget to ‘#include <cstdint>’?
          5 | #include <cassert>
        +++ |+#include <cstdint>
          6 |
      youtokentome/cpp/utf8.h:11:1: error: ‘uint32_t’ does not name a type
         11 | uint32_t chars_to_utf8(const char* begin, uint64_t size, uint64_t* utf8_len);
            | ^~~~~~~~
      youtokentome/cpp/utf8.h:11:1: note: ‘uint32_t’ is defined in header ‘<cstdint>’; did you forget to ‘#include <cstdint>’?
      youtokentome/cpp/utf8.h:13:6: error: variable or field ‘utf8_to_chars’ declared void
         13 | void utf8_to_chars(uint32_t x, std::back_insert_iterator<std::string> it);
            |      ^~~~~~~~~~~~~
      youtokentome/cpp/utf8.h:13:20: error: ‘uint32_t’ was not declared in this scope
         13 | void utf8_to_chars(uint32_t x, std::back_insert_iterator<std::string> it);
            |                    ^~~~~~~~
      youtokentome/cpp/utf8.h:13:20: note: ‘uint32_t’ is defined in header ‘<cstdint>’; did you forget to ‘#include <cstdint>’?
      youtokentome/cpp/utf8.h:13:71: error: expected primary-expression before ‘it’
         13 | void utf8_to_chars(uint32_t x, std::back_insert_iterator<std::string> it);
            |                                                                       ^~
      youtokentome/cpp/utf8.h:15:43: error: ‘uint32_t’ was not declared in this scope
         15 | std::string encode_utf8(const std::vector<uint32_t> &utext);
            |                                           ^~~~~~~~
      youtokentome/cpp/utf8.h:15:43: note: ‘uint32_t’ is defined in header ‘<cstdint>’; did you forget to ‘#include <cstdint>’?
      youtokentome/cpp/utf8.h:15:51: error: template argument 1 is invalid
         15 | std::string encode_utf8(const std::vector<uint32_t> &utext);
            |                                                   ^
      youtokentome/cpp/utf8.h:15:51: error: template argument 2 is invalid
      youtokentome/cpp/utf8.h:17:13: error: ‘uint32_t’ was not declared in this scope
         17 | std::vector<uint32_t> decode_utf8(const char *begin, const char *end);
            |             ^~~~~~~~
      youtokentome/cpp/utf8.h:17:13: note: ‘uint32_t’ is defined in header ‘<cstdint>’; did you forget to ‘#include <cstdint>’?
      youtokentome/cpp/utf8.h:17:21: error: template argument 1 is invalid
         17 | std::vector<uint32_t> decode_utf8(const char *begin, const char *end);
            |                     ^
      youtokentome/cpp/utf8.h:17:21: error: template argument 2 is invalid
      youtokentome/cpp/utf8.h:19:13: error: ‘uint32_t’ was not declared in this scope
         19 | std::vector<uint32_t> decode_utf8(const std::string &utf8_text);
            |             ^~~~~~~~
      youtokentome/cpp/utf8.h:19:13: note: ‘uint32_t’ is defined in header ‘<cstdint>’; did you forget to ‘#include <cstdint>’?
      youtokentome/cpp/utf8.h:19:21: error: template argument 1 is invalid
         19 | std::vector<uint32_t> decode_utf8(const std::string &utf8_text);
            |                     ^
      youtokentome/cpp/utf8.h:19:21: error: template argument 2 is invalid
      youtokentome/cpp/utf8.h:33:3: error: ‘uint32_t’ does not name a type
         33 |   uint32_t operator*() {
            |   ^~~~~~~~
      youtokentome/cpp/utf8.h:33:3: note: ‘uint32_t’ is defined in header ‘<cstdint>’; did you forget to ‘#include <cstdint>’?
      youtokentome/cpp/utf8.h:43:3: error: ‘uint64_t’ does not name a type
         43 |   uint64_t get_utf8_len() {
            |   ^~~~~~~~
      youtokentome/cpp/utf8.h:43:3: note: ‘uint64_t’ is defined in header ‘<cstdint>’; did you forget to ‘#include <cstdint>’?
      youtokentome/cpp/utf8.h:53:3: error: ‘uint32_t’ does not name a type
         53 |   uint32_t code_point = 0;
            |   ^~~~~~~~
      youtokentome/cpp/utf8.h:53:3: note: ‘uint32_t’ is defined in header ‘<cstdint>’; did you forget to ‘#include <cstdint>’?
      youtokentome/cpp/utf8.h:54:3: error: ‘uint64_t’ does not name a type
         54 |   uint64_t utf8_len = 0;
            |   ^~~~~~~~
      youtokentome/cpp/utf8.h:54:3: note: ‘uint64_t’ is defined in header ‘<cstdint>’; did you forget to ‘#include <cstdint>’?
      youtokentome/cpp/utf8.h: In member function ‘vkcom::UTF8Iterator vkcom::UTF8Iterator::operator++()’:
      youtokentome/cpp/utf8.h:28:14: error: ‘utf8_len’ was not declared in this scope
         28 |     begin += utf8_len;
            |              ^~~~~~~~
      youtokentome/cpp/utf8.h: In member function ‘void vkcom::UTF8Iterator::parse()’:
      youtokentome/cpp/utf8.h:61:5: error: ‘code_point’ was not declared in this scope
         61 |     code_point = chars_to_utf8(begin, end - begin, &utf8_len);
            |     ^~~~~~~~~~
      youtokentome/cpp/utf8.h:61:53: error: ‘utf8_len’ was not declared in this scope
         61 |     code_point = chars_to_utf8(begin, end - begin, &utf8_len);
            |                                                     ^~~~~~~~
      youtokentome/cpp/utf8.h:61:18: error: ‘chars_to_utf8’ was not declared in this scope
         61 |     code_point = chars_to_utf8(begin, end - begin, &utf8_len);
            |                  ^~~~~~~~~~~~~
      youtokentome/cpp/utf8.cpp: In function ‘uint32_t vkcom::chars_to_utf8(const char*, uint64_t, uint64_t*)’:
      youtokentome/cpp/utf8.cpp:73:10: error: ‘INVALID_UNICODE’ was not declared in this scope
         73 |   return INVALID_UNICODE;
            |          ^~~~~~~~~~~~~~~
      youtokentome/cpp/utf8.cpp: At global scope:
      youtokentome/cpp/utf8.cpp:111:18: error: ambiguating new declaration of ‘std::vector<unsigned int> vkcom::decode_utf8(const char*, const char*)’
        111 | vector<uint32_t> decode_utf8(const char* begin, const char* end) {
            |                  ^~~~~~~~~~~
      youtokentome/cpp/utf8.h:17:23: note: old declaration ‘int vkcom::decode_utf8(const char*, const char*)’
         17 | std::vector<uint32_t> decode_utf8(const char *begin, const char *end);
            |                       ^~~~~~~~~~~
      youtokentome/cpp/utf8.cpp: In function ‘std::vector<unsigned int> vkcom::decode_utf8(const char*, const char*)’:
      youtokentome/cpp/utf8.cpp:117:23: error: ‘INVALID_UNICODE’ was not declared in this scope
        117 |     if (code_point != INVALID_UNICODE) {
            |                       ^~~~~~~~~~~~~~~
      youtokentome/cpp/utf8.cpp: At global scope:
      youtokentome/cpp/utf8.cpp:130:18: error: ambiguating new declaration of ‘std::vector<unsigned int> vkcom::decode_utf8(const std::string&)’
        130 | vector<uint32_t> decode_utf8(const string& utf8_text) {
            |                  ^~~~~~~~~~~
      youtokentome/cpp/utf8.h:19:23: note: old declaration ‘int vkcom::decode_utf8(const std::string&)’
         19 | std::vector<uint32_t> decode_utf8(const std::string &utf8_text);
            |                       ^~~~~~~~~~~
      youtokentome/cpp/utf8.cpp: In function ‘std::vector<unsigned int> vkcom::decode_utf8(const std::string&)’:
      youtokentome/cpp/utf8.cpp:131:21: error: could not convert ‘vkcom::decode_utf8((& utf8_text)->std::__cxx11::basic_string<char>::data(), ((& utf8_text)->std::__cxx11::basic_string<char>::data() + ((sizetype)(& utf8_text)->std::__cxx11::basic_string<char>::size())))’ from ‘int’ to ‘std::vector<unsigned int>’
        131 |   return decode_utf8(utf8_text.data(), utf8_text.data() + utf8_text.size());
            |          ~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            |                     |
            |                     int
      error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for youtokentome
Failed to build youtokentome
ERROR: Could not build wheels for youtokentome, which is required to install pyproject.toml-based projects

OS: Ubuntu 23.10 x64
Python: 3.11.6
GCC: 13.2.0
Cython: 3.0.10 already installed in the venv

@WiegerWolf
Copy link

this cdbrendel@4377472 works

@WiegerWolf WiegerWolf linked a pull request Aug 22, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants