Bamboo Authentication Discussion

[rgaudin]

Please take this topic as a starting point for the definition of the Data Access Policy that we would like to see in Bamboo.
I think we all agree that the current situation is not sustainable in production:

• Datasets IDs are not available.
• Formhub does give public access to the bamboo dataset ID if linked.
• Bamboo is not available over SSL.
• Bamboo has no restriction on access: Knowing dataset ID and API is sufficient.
• Bamboo has no notion of write permission. Anybody with access can update, delete and add data.
• Bamboo has no public/private distinction.

First, I believe we need to define what we want to be able to do ;

Please note that I am assuming that:

• bamboo is a data store service
• a bamboo “customer” is the owner of one or multiple datasets.
• dataset owner can manage his own dataset's access.

One thing I really like about the current situation is that it's super simple to use, and anonymous.
While we definitely need to offer private and secure dataset, I believe we should keep the unsecure, open and easy way.

Thus I'd like to propose 3 layers of security:

• Anonymous: Doesn't require credentials, tied to a global Anonymous profile (quota, etc)
• Identified: Requires an API Key, ties the requests to it.
• Authenticated: Requires an API Key+Secret (OAuth probably)

Thoughts (not in order):

Switch to SSL only

This is not directly related to authentication but since we want to protect data, first step is probably to disallow clear HTTP.
It is possible to have both together but that's not the point right?
It's just a little setup, a little money ($70/y on godaddy) and some communication to existing users.

User Management

We want to know who owns a dataset.
This will allow us to contact the owner if needed (IO too high, data corrupted, scheduled maintenance, etc) and also allow us to present/group all datasets tied to a particular user.

For this authentication, we will need a classical auth mechanism, like a custom user/pass system or use a third party (google, github, facebook, OpenID). It might be interesting to have formhub here as the identity provider.

I'm in favor of externalizing this.

A User account would provide:

• User-level API Key
• User-level OAuth credentials
• Ability to generate per-dataset Keys and OAuth credentials (for distribution).
• Ability to revoke Keys or OAuth credentials (User-level and Dataset-level)

Identified Users

We want to know who's using a dataset.
This is usually done through simple API Keys. The owner of a DS provides API Keys to self or other clients accessing the data.
Using this would permit: quota, ban, statistics based on data consumers.
It's interesting and easy to implement.
I don't know how we would perform unit tests though if we require one (no anonymous).

Anonymous User

Anonymous (no Key provided) would simply default to a special unique Identified User name Anonymous.
All anonymous requests would share the same profile.

Authenticated Users

Authentication would need to be secure and allow third-parties (FH, jarvis-bi?) to consume the data too.
After some research I believe the best & easiest way to do it is to be an OAuth 1.0a Provider.
On the client side, it means that clients will need to use an OAuth library.
On the server side, using a library such as oauthlib (which seems like the best fit) still requires a good amount of work.

Datasets Permissions

We want dataset owner to set permissions.
Just like we can give per-user permissions to forms on Formhub, dataset owners should be able to give permissions to users.
Main differences:

• Permission is not for a User but for a Consumer (Anonymous, Identified Key or Authenticated Key).
• Permission is not global but per-endpoint
• Permission is not READ or READ-WRITE, it is Rest-oriented: GET, PUT, POST, DELETE
• Since this would be hell to setup manually, UI/API should provide templates/profile for this.

Implementation Tasks

• Database for storing Users, Keys & Permissions.
• User Login U.I
• User Management UI + API
  • Create/Revoke Keys and OAuth credentials
• Implement OAuth Server
• Implement identification/authentication in every controller.
• Implement identification + authentication in pybamboo (requests already supports OAuth)

Please send-in your comments ; am I missing something?

[mberg]

+1 on moving to SSL

I'd like to propose using formhub users for the login now. if we want to users bamboo to do basic dashboards for FH datasets each user will need to have a bamboo api key anyways. I think this is somewhat strategic too. FH bamboo can be tied to our client work and we can keep bamboo more as a pure / agnostic programming platform. I don't think we're at the point to offer a standalone webservice for bamboo right now.

We can use the authentication for other types of API needs for FH too (correct me if i'm wrong here).

Will comment more on permissions and stuff after I've had more time to review this.

Thanks Renaud this is a great start.

[chrisnatali]

Our OSM instance might be able to tap into this as well (it already supports OAuth and OpenID).

[pld]

This is great, thanks @rgaudin. I agree with @mberg that we should use FH for now, then we can overlook the UI tasks and expect FH to be responsible for this. I am also not convinced that we are the point where it is worthwhile to implement a sophisticated permission system, not that we wouldn't want this in the future, but I wonder if this is something current users require in the near-term. Perhaps, for now, we can lean on FH for most of this.

[rgaudin]

Well, I'm not sure I understand well what you guys want ; neither I do about what leaning on FH exactly means in practice. Given zero direction, I've made a few propositions so I expect a little more guidance than it's not worthwhile :)

Password Keys

Talking to some of you, it seems like what you want is just a binary way to make a dataset private, like we're doing with Formhub forms.
Also, I understand that you want to deal with it only in fomhub but I can't see how this can be secure so here's what we could do:

• On Dataset creation, or using a new endpoint like /datasets/xxx/private (PUT), you would be able to set a dataset private or public. The request will return a key which acts as a password.
• Every request would now require a specific HTTP Header containing the Key.
• Changing the status of the dataset would require the Key (we have no notion of owner)
• Key can be stored in the dataset meta-data.
• This would require SSL.

Formhub could consume the simple entry point described above and thus, in fine, provide the keys in a unified U.I.

If you like this proposition, let me know and I'll make some research on how to make this the most standard (haven't check but that reminds me of HTTP Basic Auth).

Of Course, this is not future proof but it could serve your immediate needs. I'm worried though that going this route, will soon raise new needs like permissions, not whole data, etc. and then we will have to rediscuss all this

Feature Poll

Maybe it's easier if each of you can just copy-paste the following table with your opinion:

Prop	YES | NO | LATER	Comment
SSH Only		@mberg needs to buy the cert.
User Management		github | Google | Formhub (how? OpenID?)
Identified Users		Should the keys be emitted by bamboo or formhub as suggested by Matt?
Anonymous User		Ability to use bamboo as we're doing it now.
Authenticated Users		OAuth? Formhub? (then how?)
Permissions (fine)
Permissions (simple)		Please suggest what kind
Password Keys		See above
Bamboo-side API to manage Keys/Perms
Bamboo-side U.I to manage Keys/Perms

@pld @ukanga @larryweya @mejymejy @mberg

[ukanga]

Prop	YES | NO | LATER	Comment
SSH Only	LATER	@mberg needs to buy the cert.
User Management	YES | OpenID | formhub	github | Google | Formhub (how? OpenID?)
Identified Users	YES | formhub	Should the keys be emitted by bamboo or formhub as suggested by Matt?
Anonymous User	YES	Ability to use bamboo as we're doing it now.
Authenticated Users	OAuth	OAuth? Formhub? (then how?)
Permissions (fine)	LATER
Permissions (simple)	YES| will go with your earlier suggestions	Please suggest what kind
Password Keys	NO	See above
Bamboo-side API to manage Keys/Perms	YES | unless there is a way to handle this in formhub, sort of formhub to act as the bamboo front end
Bamboo-side U.I to manage Keys/Perms	see previous row

[larryweya]

My feeling is that when bamboo does go into production we will want to

1. Know which user owns which datasets. So for things like service plans we will want to know how much storage a particular user is using, how many API calls etc.
2. Allow the owner to manage the dataset(s) permissions - allow public availability, or allow specific users access. When we start talking about specific users, we definitely need to be able to identify them.

I dont think we should implement one authentication method and implement another at a later stage. Based on our current use cases I think we should implement user management now.

That said, I like how the bamboo API is exposed and I think we should implement the permission assignment etc. using the same kind of REST API as opposed to exposing a UI. The UI can be built off the pybamboo and bamboo.js APIs as needed.

Since at the moment bamboo users will primarily be formhub users, it makes sense to authenticate them using current formhub users. We should however keep in mind that at some point in the future we will likely have users without formhub accounts who want access to bamboo. Therefore leaving room for other authentication providers makes sense which also makes a case for a bamboo based user management implementation.

I'm however not sure that you need to implement an oath server on bamboo's end, depedning how open we want to be, we can allow a certain list of providers, formhub being one.

So what I'm proposing, implementation wise is:

• Database for storing Users, Keys & Permissions.
• User Login U.I
• User Management UI + API
  • Create/Revoke Keys and OAuth credentials
• Implement OAuth Server
• Implement identification/authentication in every controller.
• Implement identification + authentication in pybamboo (requests already supports OAuth)

Prop	YES | NO | LATER	Comment
SSH Only	YES	@mberg needs to buy the cert.
User Management	YES	github | Google | Formhub (how? OpenID?)
Identified Users	YES formhub	Should the keys be emitted by bamboo or formhub as suggested by Matt?
Anonymous User	NO	Ability to use bamboo as we're doing it now.
Authenticated Users	OAuth server on formhub	OAuth? Formhub? (then how?)
Permissions (fine)		Dont get the difference between fine and simple
Permissions (simple)		Please suggest what kind
Password Keys	NO	I'd prefer if a dataset's owner generates and distributes keys
Bamboo-side API to manage Keys/Perms	YES
Bamboo-side U.I to manage Keys/Perms	NO

[pld]

I have never been a bamboo client so I do not have strong opinions.
Whatever we decide it should be based on use cases, Larry did a great job
describing those. I agree with and second everything Larry wrote, count my
votes the same his (but perhaps weighted less).

[myf]

Here are my thoughts on permissions and authetications:

• I like the three levels of user behavior as @rgaudin suggested
• I really like the idea that anonymous users have access to bamboo while ensuring some level of management (i.e. their database will not be deleted by other anons if they want to keep it)
• I like to keep the management issue simple on bamboo side with API
• so interms of accessability, I propose
  • PUT and GET should be defualt open w/o API key
  • DELETE can only be issued by the creater with an API key
  • creater can also change the accesibility level of PUT and GET but generating an api key and granting others accessibility by issueing these keys

Feature Poll

Prop	YES | NO | LATER	Comment
SSH Only	Yes	@mberg needs to buy the cert.
User Management	Sort of	github | Google | Formhub (how? OpenID?)
Identified Users	No	Should the keys be emitted by bamboo or formhub as suggested by Matt?
Anonymous User	Yes	Ability to use bamboo as we're doing it now.
Authenticated Users	No	OAuth? Formhub? (then how?)
Permissions (fine)
Permissions (simple)	Yes	Please suggest what kind
Password Keys	Yes API keys	See above
Bamboo-side API to manage Keys/Perms	No
Bamboo-side U.I to manage Keys/Perms	No

[mejymejy]

first, i am with @renaud in that it is unclear to me what the real desire is in terms of functional outputs and where they should be implemented. i believe that we have a lot of ideas about what we want to do (which is awesome) and this is going to be a matter of priorities and vision. if the driving force is for formhub to use bamboo and expose some of its functionality, then there are several ways to achieve that including some that involve little to no actual bamboo development. if we are talking about what we would like the bamboo.io web service to be, then that is something slightly different. i think what is going on is that we are trying to talk about all of these things at the same time which makes it hard to move forward with development. what we should do is clarify where we want to go and the setup that we see as sustainable and then we can make sure that our work in the near term fits into our overall progression.

from what i can tell, we would like to use bamboo in formhub while at the same time realizing that formhub is just one user of bamboo. it would make sense to add functionality to bamboo such that any other service could use it in the same way. here are some feelings about this that i think inform my answers to the questions posed:

• the anonymous use of bamboo is cool and something that we should preserve
• no one will really use/trust bamboo if there are no restrictions on CRUD operations on data
• we should work to maintain bamboo as a generic tool that will support many use cases
• it is helpful to keep in mind that the core bamboo can be separate from a hosted webservice (bamboo.io) which is not really the case right now
• i am hesitant to put users into bamboo as it is now, but if we do it i think we should go all in and possibly split the backend bamboo service from a front end bamboo manager thingy

Prop	YES | NO | LATER	Comment
SSH Only	SSL? YES	makes sense for any data and is simpler in many regards if used by default
User Management	YES	but separate from bamboo core
Identified Users	YES	thouhgh not sure this should be independent from Authenticated
Anonymous User	YES	though there will need to be restrictions/quota not sure we can enforce security unless it is "Identfied User"
Authenticated Users	YES	probably everything dealing with actual datasets, OAuth2
Permissions (fine)	NO	taking this to mean "too complicated"
Permissions (simple)	YES	should be simple
Password Keys	YES	API keys
Bamboo-side API to manage Keys/Perms	YES
Bamboo-side U.I to manage Keys/Perms	NO	not needed at first, can be built using bamboo libs

we really need to discuss this over skype or google hangout to get through it...

[rgaudin]

Guys, this is great and I agree with what you all said.
As suggested, I think we need a real talk about what path we want to take.
When should this happen? @prabhas ?

On Thu, Mar 14, 2013 at 5:44 PM, Mark Johnston [email protected]:

first, i am with @renaud https://github.com/renaud in that it is
unclear to me what the real desire is in terms of functional outputs and
where they should be implemented. i believe that we have a lot of ideas
about what we want to do (which is awesome) and this is going to be a
matter of priorities and vision. if the driving force is for formhub to use
bamboo and expose some of its functionality, then there are several ways to
achieve that including some that involve little to no actual bamboo
development. if we are talking about what we would like the bamboo.io web
service to be, then that is something slightly different. i think what is
going on is that we are trying to talk about all of these things at the
same time which makes it hard to move forward with development. what we
should do is clarify where we want to go and the setup that we see as
sustainable and then we can make sure that our work in the near term fits
into our ov erall progression.

from what i can tell, we would like to use bamboo in formhub while at the
same time realizing that formhub is just one user of bamboo. it would make
sense to add functionality to bamboo such that any other service could use
it in the same way. here are some feelings about this that i think inform
my answers to the questions posed:

• the anonymous use of bamboo is cool and something that we should
  preserve

• no one will really use/trust bamboo if there are no restrictions on
  CRUD operations on data

• we should work to maintain bamboo as a generic tool that will
  support many use cases

• it is helpful to keep in mind that the core bamboo can be separate
  from a hosted webservice (bamboo.io) which is not really the case
  right now

• i am hesitant to put users into bamboo as it is now, but if we do it
  i think we should go all in and possibly split the backend bamboo service
  from a front end bamboo manager thingy

  Prop YES | NO | LATER Comment SSH Only SSL? YES makes sense for any
  data and is simpler in many regards if used by default User Management
  YES but separate from bamboo core Identified Users YES thouhgh not sure
  this should be independent from Authenticated Anonymous User YES though
  there will need to be restrictions/quota not sure we can enforce security
  unless it is "Identfied User" Authenticated Users YES probably
  everything dealing with actual datasets, OAuth2 Permissions (fine) NO taking
  this to mean "too complicated" Permissions (simple) YES should be simple Password
  Keys YES API keys Bamboo-side API to manage Keys/Perms YES Bamboo-side
  U.I to manage Keys/Perms NO not needed at first, can be built using
  bamboo libs

we really need to discuss this over skype or google hangout to get through
it...

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/490#issuecomment-14917483
.

[prabhasp]

Monday morning before or after the dev meeting? Tuesday morning 9am EST?
Sorry, I haven't weighed in because I don't fully understand everything in
the proposal; will try to, and vote, by end of day.

On Thu, Mar 14, 2013 at 3:22 PM, rgaudin [email protected] wrote:

Guys, this is great and I agree with what you all said.
As suggested, I think we need a real talk about what path we want to take.
When should this happen? @prabhas ?

On Thu, Mar 14, 2013 at 5:44 PM, Mark Johnston [email protected]:

first, i am with @renaud https://github.com/renaud in that it is
unclear to me what the real desire is in terms of functional outputs and
where they should be implemented. i believe that we have a lot of ideas
about what we want to do (which is awesome) and this is going to be a
matter of priorities and vision. if the driving force is for formhub to
use
bamboo and expose some of its functionality, then there are several ways
to
achieve that including some that involve little to no actual bamboo
development. if we are talking about what we would like the bamboo.ioweb
service to be, then that is something slightly different. i think what
is
going on is that we are trying to talk about all of these things at the
same time which makes it hard to move forward with development. what we
should do is clarify where we want to go and the setup that we see as
sustainable and then we can make sure that our work in the near term
fits
into our ov erall progression.

from what i can tell, we would like to use bamboo in formhub while at
the
same time realizing that formhub is just one user of bamboo. it would
make
sense to add functionality to bamboo such that any other service could
use
it in the same way. here are some feelings about this that i think
inform
my answers to the questions posed:

• the anonymous use of bamboo is cool and something that we should
  preserve
• no one will really use/trust bamboo if there are no restrictions on
  CRUD operations on data
• we should work to maintain bamboo as a generic tool that will
  support many use cases
• it is helpful to keep in mind that the core bamboo can be separate
  from a hosted webservice (bamboo.io) which is not really the case
  right now
• i am hesitant to put users into bamboo as it is now, but if we do it
  i think we should go all in and possibly split the backend bamboo
  service
  from a front end bamboo manager thingy

Prop YES | NO | LATER Comment SSH Only SSL? YES makes sense for any
data and is simpler in many regards if used by default User Management
YES but separate from bamboo core Identified Users YES thouhgh not sure
this should be independent from Authenticated Anonymous User YES though
there will need to be restrictions/quota not sure we can enforce
security
unless it is "Identfied User" Authenticated Users YES probably
everything dealing with actual datasets, OAuth2 Permissions (fine) NO
taking
this to mean "too complicated" Permissions (simple) YES should be simple
Password
Keys YES API keys Bamboo-side API to manage Keys/Perms YES Bamboo-side
U.I to manage Keys/Perms NO not needed at first, can be built using
bamboo libs

we really need to discuss this over skype or google hangout to get
through
it...

—
Reply to this email directly or view it on GitHub<
https://github.com/modilabs/bamboo/issues/490#issuecomment-14917483>
.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/490#issuecomment-14923581
.

Prabhas Pokharel
http://prabhasp.com
US mobile: +1 347 948 7654
twitter/skype/facebook/whatever: prabhasp