Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Network schema io #44

Merged
merged 64 commits into from
Jun 13, 2017
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
ec2c0b9
Merge pull request #37 from markplagge/Develop
Apr 14, 2017
8a201b5
Adding json spike to CSV converter and NeMo run script
Apr 17, 2017
c3ba33e
Quick spike save function
Apr 17, 2017
9581297
Some pythonic changes to make the JSON reader more efficient.
Apr 18, 2017
98cfeb1
added progressbar
Apr 18, 2017
d7f2721
fixed mp error
Apr 18, 2017
d411bf8
changed to future rather than MP
Apr 18, 2017
2655c1a
Read TN JSON spikes and faster mp
Apr 18, 2017
ceadaca
Forgot result call.
Apr 18, 2017
a9e7180
future calls to future calls are bad when using JIT
Apr 18, 2017
c19eb1b
Fixed issues. to_csv is still slow.
Apr 18, 2017
ba7518e
clean cruft
Apr 18, 2017
fb6c014
Added JIT back
Apr 18, 2017
b392865
Quick changes before adding tqdm and fixing neuron params.
Apr 18, 2017
dd2f2a0
qc change
Apr 18, 2017
068c7c4
Python JSON->CSV working with spikes
Apr 19, 2017
2ebffe2
More workable - CLI non functional.
Apr 19, 2017
3de1176
Minor typo in testIO.h
Apr 19, 2017
e7424d9
more typos
Apr 19, 2017
c7516e7
more script fixes.
Apr 19, 2017
2ccf839
typo
Apr 19, 2017
18efd33
more minor changes
Apr 19, 2017
42b7c03
TN scripts working, added some test csv files.
Apr 21, 2017
6422715
some minor tweaks to neuron reader
Apr 24, 2017
3d2f97b
IO mostly working - need to check neuron CSV read validity
Apr 25, 2017
4914c8a
IO Mostly working
Apr 25, 2017
b1ddec6
Added basic neuron stat monitor for sanity check
Apr 25, 2017
fac82c5
Changed default network structure for easier testing.
Apr 25, 2017
b5a1649
fixed malloc/free error.
Apr 25, 2017
f7474fa
Minor list comm typo.
Apr 25, 2017
533aeaa
fixed net out code. Need to start doing sanity checks on model CSV
Apr 25, 2017
6843e4b
Fixed, but CSV input to Neuron NF
Apr 26, 2017
895f725
Some quick changes before major retooling
Apr 28, 2017
9fca7f1
network read is still buggy - not loading destinations properly.
Apr 28, 2017
c601030
IO stack update
May 1, 2017
ff806ac
new tn format - using a more structured format will make debugging ea…
May 1, 2017
e876896
New, better, file format.
May 1, 2017
3f8884e
new nemo reader - minor bugs
May 1, 2017
47ded1f
nemo python with new file def working.
May 1, 2017
f7b9d5e
small bug fix on json reader
May 2, 2017
faeb07e
minor JSON parser changes.
May 2, 2017
9de119e
added neuron name conversion function
nmcglo May 3, 2017
4fa868d
fixed getArrayFromMulti
nmcglo May 4, 2017
7d92f11
fixed populateArray() position bug
nmcglo May 4, 2017
3606069
fixed hardcoded default num cores
nmcglo May 4, 2017
de0ef53
Merge pull request #39 from markplagge/network-schema-io-neil
May 4, 2017
b4497f1
NeMo Model CFG Reader framework written
May 4, 2017
0538179
New Build config with partial support for reading in config file.
May 4, 2017
03c1bf4
Builds but arrays have bugs.
May 4, 2017
21c737f
Arrray bug
May 4, 2017
7c13cc0
L Up -// tn_json_reader up.
May 5, 2017
6a1b74e
Latest NS IO
May 5, 2017
0324aa9
Found spiking / non spiking issue
May 6, 2017
cead8d5
Added MNIST network def to script test pool
May 7, 2017
187b810
fixed CMAKE build process with LUA
May 8, 2017
25b6d95
Fixed CMAKE build process with LUA settings
May 8, 2017
47ddb20
Removed extra print statement
May 8, 2017
fe85469
Changed input filenames to compile-time option.
May 11, 2017
7913bb9
added some error handling for model reads
May 11, 2017
fb2906c
Minor bug fixes
May 14, 2017
00961ea
Removed c11 flags, replaced with gnu99 flags for build.
May 14, 2017
222ee35
added comparing script and example model
nmcglo May 21, 2017
c44477e
Added Compare Script Arguments
nmcglo May 21, 2017
f40e06a
Ready to debug
Jun 13, 2017
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -51,3 +51,9 @@ cmake-build-debug/
cmake-build-release/src/

cmake-build-release/

scripts/ol_api/t1\.csv

scripts/tn_api/__init__.pyc

*.pyc
4 changes: 4 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,7 @@
path = src/lib/c11t
url = https://github.com/jtsiomb/c11threads.git

[submodule "lua"]
path = lua
url = https://github.com/markplagge/lua.git
branch = lua-5.1
100 changes: 100 additions & 0 deletions .vscode/.ropeproject/config.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
# The default ``config.py``
# flake8: noqa


def set_prefs(prefs):
"""This function is called before opening the project"""

# Specify which files and folders to ignore in the project.
# Changes to ignored resources are not added to the history and
# VCSs. Also they are not returned in `Project.get_files()`.
# Note that ``?`` and ``*`` match all characters but slashes.
# '*.pyc': matches 'test.pyc' and 'pkg/test.pyc'
# 'mod*.pyc': matches 'test/mod1.pyc' but not 'mod/1.pyc'
# '.svn': matches 'pkg/.svn' and all of its children
# 'build/*.o': matches 'build/lib.o' but not 'build/sub/lib.o'
# 'build//*.o': matches 'build/lib.o' and 'build/sub/lib.o'
prefs['ignored_resources'] = ['*.pyc', '*~', '.ropeproject',
'.hg', '.svn', '_svn', '.git', '.tox']

# Specifies which files should be considered python files. It is
# useful when you have scripts inside your project. Only files
# ending with ``.py`` are considered to be python files by
# default.
#prefs['python_files'] = ['*.py']

# Custom source folders: By default rope searches the project
# for finding source folders (folders that should be searched
# for finding modules). You can add paths to that list. Note
# that rope guesses project source folders correctly most of the
# time; use this if you have any problems.
# The folders should be relative to project root and use '/' for
# separating folders regardless of the platform rope is running on.
# 'src/my_source_folder' for instance.
#prefs.add('source_folders', 'src')

# You can extend python path for looking up modules
#prefs.add('python_path', '~/python/')

# Should rope save object information or not.
prefs['save_objectdb'] = True
prefs['compress_objectdb'] = False

# If `True`, rope analyzes each module when it is being saved.
prefs['automatic_soa'] = True
# The depth of calls to follow in static object analysis
prefs['soa_followed_calls'] = 0

# If `False` when running modules or unit tests "dynamic object
# analysis" is turned off. This makes them much faster.
prefs['perform_doa'] = True

# Rope can check the validity of its object DB when running.
prefs['validate_objectdb'] = True

# How many undos to hold?
prefs['max_history_items'] = 32

# Shows whether to save history across sessions.
prefs['save_history'] = True
prefs['compress_history'] = False

# Set the number spaces used for indenting. According to
# :PEP:`8`, it is best to use 4 spaces. Since most of rope's
# unit-tests use 4 spaces it is more reliable, too.
prefs['indent_size'] = 4

# Builtin and c-extension modules that are allowed to be imported
# and inspected by rope.
prefs['extension_modules'] = []

# Add all standard c-extensions to extension_modules list.
prefs['import_dynload_stdmods'] = True

# If `True` modules with syntax errors are considered to be empty.
# The default value is `False`; When `False` syntax errors raise
# `rope.base.exceptions.ModuleSyntaxError` exception.
prefs['ignore_syntax_errors'] = False

# If `True`, rope ignores unresolvable imports. Otherwise, they
# appear in the importing namespace.
prefs['ignore_bad_imports'] = False

# If `True`, rope will insert new module imports as
# `from <package> import <module>` by default.
prefs['prefer_module_from_imports'] = False

# If `True`, rope will transform a comma list of imports into
# multiple separate import statements when organizing
# imports.
prefs['split_imports'] = False

# If `True`, rope will sort imports alphabetically by module name
# instead of alphabetically by import statement, with from imports
# after normal imports.
prefs['sort_imports_alphabetically'] = False


def project_opened(project):
"""This function is called after opening the project"""
# Do whatever you like here!
Binary file added .vscode/.ropeproject/objectdb
Binary file not shown.
3 changes: 3 additions & 0 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
// Place your settings in this file to overwrite default and user settings.
{
}
48 changes: 43 additions & 5 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,22 +1,60 @@
cmake_minimum_required(VERSION 2.6)
cmake_minimum_required(VERSION 2.8)
set(CMAKE_C_STANDARD 99)
ADD_COMPILE_OPTIONS(-std=gnu99)
#set(CMAKE_C_FLAGS "${CMAKE_C_CFLAGS} -std=gnu99")
project(SuperNeMo)
enable_testing()
include(ExternalProject)


set (RD "${CMAKE_CURRENT_SOURCE_DIR}/ROSS/")
set (ND "${CMAKE_CURRENT_SOURCE_DIR}/src/")
set (LuaD "${CMAKE_CURRENT_SOURCE_DIR}/lua")


#cmake_policy(SET CMP0012 OLD)
# LUA Options
#SET(USE_LUA_JIT FALSE BOOL)
option(USE_LUA_JIT "Use JIT lua project for scripting" OFF)
#SET(USE_LUA_JIT TRUE)
ADD_SUBDIRECTORY(${RD})

if(USE_LUA_JIT)
message(STATUS "LUA JIT PROJECT" )

else()
ADD_SUBDIRECTORY(${LuaD})
include_directories(${MPI_INCLUDE_PATH}
${LuaD}/src
${lua_BINARY_DIR})
endif()


ADD_SUBDIRECTORY(${ND})
find_package(MPI REQUIRED)
include_directories(${MPI_INCLUDE_PATH})
#find_package(MPI REQUIRED)



SET(CMAKE_C_COMPILER mpicc)
SET(CMAKE_CXX_COMPILER mpicxx)
#get_target_property(CFLGS ROSS CMAKE_C_FLAGS)
set_target_properties( NeMo
PROPERTIES
ARCHIVE_OUTPUT_DIRECTORY "${CMAKE_BINARY_DIR}/lib"
LIBRARY_OUTPUT_DIRECTORY "${CMAKE_BINARY_DIR}/lib"
RUNTIME_OUTPUT_DIRECTORY "${CMAKE_BINARY_DIR}/bin"
CONFIG_OUTPUT_DIRECTORY "${CMAKE_BINARY_DIR}/scripts"
USE_LUA_JIT ${USE_LUA_JIT}
# CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${CFLGS} -std=gnu11"
)


#files for running IO:
#


configure_file("demo.csv" "${CMAKE_BINARY_DIR}/bin/demo.csv" COPYONLY)
CONFIGURE_FILE(${CMAKE_CURRENT_SOURCE_DIR}/src/IO/model_read.lua ${CMAKE_CURRENT_BINARY_DIR}/bin/model_read.lua COPYONLY)
CONFIGURE_FILE(${CMAKE_CURRENT_SOURCE_DIR}/src/IO/tn_types.lua ${CMAKE_CURRENT_BINARY_DIR}/bin/tn_types.lua COPYONLY)



#TODO: Add targets to the build-tree export set
1 change: 1 addition & 0 deletions line.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
TN,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,1,0,0,0,0,0,0,0,0,1,0,1,0,0,1,0,1,0,0,0,0,0,1,0,1,0,0,0,0,0,1,0,0,0,0,0,1,1,0,0,0,1,0,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,1,0,0,0,0,0,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1,3,2,1,0,0,0,0,
1 change: 1 addition & 0 deletions lua
Submodule lua added at d2e7e7
31 changes: 31 additions & 0 deletions quick_nemo_input_ref.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
Fields:
type
coreID
localID
//Arrays are first //
synapticConnectivity
SigmaGI
S
weight_selection
// synaptic_weights calculated value
epsilon
sigma_l
lambda
c
posThreshold
negThreshold
thresholdRPNMask
VR
sigmaVR
resetVoltage (VR)
resetMode
kappa
omega
signalDelay
destCore
destLocal
//CUSTOM FOR NEMO //
isOutputNeuron
isSelfFiring
isActiveNeuron (true) //////not used!

101 changes: 0 additions & 101 deletions scripts/api_gen.py

This file was deleted.

65 changes: 0 additions & 65 deletions scripts/api_gen2.py

This file was deleted.

Loading