network_inventory/docs/notes.org

#+TITLE: Network Inventory
:preamble:
#+author: Andreas Zweili
:end:

* Introduction
** How to work with this file

1. All URLs should get included as footnotes on order to have a cleaner layout.
2. All tasks need a priority, tasks with a priority higher than C are
   considered must haves for the 1.0 release. Everything with C or lower is
   considered a nice to have an might never get implemented.
3. Once an item changes its status to either DONE or CANCELLED it gets moved to
   the "Done" heading. CANCELLED items should get an explanation why they were
   cancelled.

** Vocabulary

- Technician :: An employee working with the inventory tool and full access to
  its information.
- Customer :: A customer which owns some of the devices in the inventory tool
  and might have access to only the information related to the devices he owns.

* TODO Must Have [0/14]
** TODO [#A] Forms [0/12]
*** TODO Computer Forms [0/3]
- [X] Add
- [X] Update
- [ ] Delete

**** NEXT CPU Forms
- [X] Add
- [X] Update
- [X] Delete

**** NEXT RAM Forms
- [X] Add
- [X] Update
- [X] Delete

**** NEXT Disk Forms
- [X] Add
- [X] Update
- [X] Delete

*** NEXT Customer Forms
- [X] Add
- [ ] Update
- [ ] Delete

*** NEXT Backup Forms
- [ ] Add
- [ ] Update
- [X] Delete

*** NEXT UserLicense Forms
- [ ] Add
- [ ] Update
- [ ] Delete

*** NEXT DeviceLicense Forms
- [X] Add
- [X] Update
- [X] Delete

*** NEXT Net Forms
- [ ] Add
- [ ] Update
- [ ] Delete

*** NEXT Software Forms
- [ ] Add
- [ ] Update
- [ ] Delete

*** NEXT Users Forms
- [ ] Add
- [ ] Update
- [ ] Delete

*** NEXT Groups Forms
- [ ] Add
- [ ] Update
- [ ] Delete

*** NEXT Limit the dropdowns in update views

When a user edits an object he should only be able to select objects from the
dropdowns he's allowed to view.
Meaning that we have to limit every dropdown in an update view to the customer
he's allowed to see. Like only the Nets or Users related to that customer.

We can use the get_objects helper function in core.utils.

*** NEXT Limit the customer views to superusers
*** NEXT Add + button for nets in the DeviceInNet form

This way we can quickly add a net when we want to add an IP to a device.

** NEXT [#A] a backup should be able to contain multiple computers   :model:
** NEXT [#A] implement view for groups
** NEXT [#A] Add a feature to copy objects

This is a must have for the first release.
As a user I would like to have a way of quickly copy an object and make some
adjustments in order to add many objects after another.

** NEXT [#A] Add search boxes to the views.

So that one can search for a string in the responding column.

** NEXT [#A] Create custom user model                                :model:

It is best practice to create a custom user model to allow future modifications
to the users without causing problems.

** NEXT [#B] Implement a license check into all forms

This should prevent technicians from assigning licenses which the customer has
already fully used.

** NEXT [#B] calculate licence usage for customer

The view should show the licenses which the customer currently has available
and how many are already used. In addition it should show a visual warning to
the technician when the limit is reached.

** NEXT [#B] Convert the NETSheet Data file to YML fixtures.

A lot of this is already done. Only the hardware models are currently missing.

** NEXT [#B] Have a look at the documentation of django-nested-admin

I implemented nested-admin currently in a very basic way. I should read the
documentation in order to make sure that I'm using it correctly.

** NEXT [#B] have a look at django select_related, it might solve a problem for me.

I often find myself trying to get related objects. The method select_related
might help with that [fn:1]:
-

** NEXT [#B] Extend the Admin tables

The admin tables show currently very little information about the various
objects. At minimum every object should display the customer it belongs to.

** NEXT [#B] In the warranty form validate the date inputs

The starting date shouldn't be allowed to be newer than the end date.

** NEXT [#C] Filter Hardware Model to corresponding device manufacturer

When changing the HardwareModel field of a device the dropdown should be
filtered to the provided DeviceManufacturer.

Currently it could still make sense to make the DeviceManufacturer only
available through the HardwareModel. This way we wouldn't have to filter the
HardwareModel dropdown. However we would loose the ability to only select the
DeviceManufacturer for a device in case we don't know the specific model which
happens quite often.

* TODO Nice to Have [0/11]
** NEXT [#C] allow technicians to add custom fields

This would allow technicians to create custom models without change
Maybe this approach would be something [fn:2]:

** NEXT [#C] calculate the used space on a host

Means calculate the size all the VMs would use if they were thick.
This could help a technician to properly plan ressources on a host.

** NEXT [#C] include a RAID calculator

I would like to use this to show the usable space in a RAID system. Currently
we enter this information by hand but it would be easier to calculate it
automatically [fn:3].
-

** NEXT [#C] Get warranty information from Dell

We sell a lot of Dell devices and it would be nice to use the service tags to
collect the warranty information directly from Dell. There's an API for that [fn:4]:

** NEXT [#C] A "to deactivate" feature on inventory users

This way a technician could mark a user for deactivation and anyone could check
if there are users to deactivate. This would help if we would've to deactivate
a user at a certain date. The inventory tool could then show to all technicians
that the user needs to be deactivated. Then any technician could deactivate the
user and not just the technician responsible for the customer, increasing the
security of the customer.

** NEXT [#C] change the admin url

For security reasons it's recommended to change the name of the admin panel
url. This way automated tools can't find it so easy. It only increases the
security slightly.

** NEXT [#C] implement guardian

This needs to be done for basically every model which lives on a view. E.g.
~BackupListView~, ~SoftwareListView~. I can’t remember how this should be
implemented. However it might be implemented in the customer table view. The
security concept works like this:
1. check if the user is logged in
2. check if the user is allowed to view the customer, if not return an error
3. Get all matching objects which the user is allowed to view. Step two can't
   be replaced by an empty table because we need a customer object to operate
   on. Therefore it's better to quickly check the customer before we fetch all
   the other objects from the database.

** NEXT [#C] Implement an excel import and export

might be achieved with the django-excel project [fn:6].
This might be a nice to have feature but the copy button is more important to
achieve a similar funcition.

** NEXT [#C] implement SoftwareDetailView

I don't remember what the initial idea here was. We could show here
which customers are using this software. But that is currently a really low
priority item.

** NEXT [#C] Implement the .all command in templates

This stackoverflow post should help [fn:9]

** NEXT [#B] Check tests for response.context[‚table‘]

This would allow for tests of the views which check explicitly what gets
returned by the view. Might be easier/faster then rendering the whole view.

However for some views it would be better to test the final view because the
template contains logic which can fail.

* Resources

[fn:11] https://gist.github.com/neara/6209563

[fn:10] https://github.com/timhughes/django-cbv-inline-formset

[fn:9] https://stackoverflow.com/questions/4270330/django-show-a-manytomanyfield-in-a-template

[fn:8] https://docs.djangoproject.com/en/3.0/ref/forms/fields/#multiplechoicefield

[fn:7] https://stackoverflow.com/questions/23059088/manytomany-field-check-if-relation-exists

[fn:6] https://github.com/pyexcel-webwares/django-excel

[fn:5] https://docs.djangoproject.com/en/2.2/topics/http/urls/#views-extra-options

[fn:4] https://www.programmableweb.com/api/dell-warranty-status-rest-api

[fn:3] https://thoughtworksnc.com/2017/08/30/writing-a-raid-calculator-in-python

[fn:2] https://stackoverflow.com/questions/34907014/django-allow-user-to-add-fields-to-model

[fn:1] https://docs.djangoproject.com/en/2.2/ref/models/querysets/#select-related

** Class Based Views

- http://ccbv.co.uk/

** Design
*** Admin themes
- django-grappelli
- django-suit
- django-admin-bootstrapped

** Forms

- https://django-crispy-forms.readthedocs.io/en/latest/index.html
- https://stackoverflow.com/questions/25321423/django-create-inline-forms-similar-to-django-admin*25340256
- https://stackoverflow.com/questions/5171365/django-inline-form-with-custom-forms

** Permissions

- https://django-guardian.readthedocs.io/en/stable/userguide/assign.html
- https://github.com/dfunckt/django-rules/blob/master/README.rst

#+begin_src python
decororator (function) :
  if user has permission(object.customer):
    return function
#+end_src

Maybe it would be possible to add a property to the classes which allows to
access the customer of an object like this:

#+begin_src python
object.customer
#+end_src

* Links to include

- https://docs.djangoproject.com/en/2.2/ref/models/querysets/#id4
- https://docs.djangoproject.com/en/2.2/ref/request-response/
- https://duckduckgo.com/?q=django+get_related&t=fpas&ia=qa
- https://pybit.es/selenium-pytest-and-django.html
- https://stackoverflow.com/questions/28533174/programatically-accessing-django-models-from-another-app
- https://stackoverflow.com/questions/54592026/how-to-create-a-custom-mixin-in-django
- https://stackoverflow.com/questions/58307055/access-django-model-name-from-admin-url-pattern
- https://stackoverflow.com/questions/6069070/how-to-use-permission-required-decorators-on-django-class-based-views#6069444

* Done
** DONE Recreate the RM in draw.io

The Dia RM is okay but not really that great. Draw.io would give a better
result.

** DONE create multiple requirements files
** DONE put passwords into environment variables
** DONE Permissions recherchieren
** DONE customer tabelle erweitern mit listen
** DONE Models erstellen
** DONE Add a Counter to the RAM Modules
** DONE Create a NET category where a device can live in.

This NET Category should display it's IP range, Subnet mask and show it's DHCP
range if one is configured.

** DONE Create class DeviceInNet

This class shows the relationship between the device and a NET. An attribute of
a DeviceInNet should be an IP address.

** DONE Create an abstract company class
** DONE Create Customer and a Manufacturer sub class Those two would be based on

the company class. I'm currently not sure how I should handle the case where a
company is both a customer and a manufacturer.

** DONE A text field next to the customer

where one can enter additional information which can't be put into the normal
documentation.

** DONE Fix test for net detail view
** DONE NETs, add a description field, for NETs like HEHImmo it might be nice to

have a short description for what it is intendet.

** DONE ComputerDetailView, add link to SoftwareDetailView
** DONE implement NETSheet list

this view should give an overview of all the devices in the NET and there
current IP Address.

** DONE implement BackupListView
** DONE Filter the queryset in the AllComputerView

so that it only shows the customers the current user is allowed to view

** DONE Disks in RAID and RAID have overlapping Felds (disks appear on both).

And they don't have the proper relationship. There can be disks from variing
sizes in a RAID therefore the relationship between DisksInRaid and
RaidInComputer needs to be a manytoone relationship

** DONE fix column name links in customer table

they throw an error when one clicks on them.

** DONE ComputerDetailView, add all properties to the view table
** DONE implement UserListView
** DONE implement SoftwareListView

this and the next view would probably better be a License view. Since the
software should be available to all devices from all customers. It doesn’t make
much sense to add 100 of different Office softwares. Probably a Software model
could be attached to a License model.

** DONE implement UserDetailView
** DONE Implement the license so that it can get attached to a user

when the user gets created. This way they might get less easily forgotten.

** DONE fix the Makefile so that the fixtures don't get applies twice.

This is already done for the ~make local~ command but needs fixing in the
~make~ command. However there's a bit more difficult because it runs in Docker
and with PostgreSQL

** DONE refactor the project to have a core app.
CLOSED: [2020-01-14 Tue 21:25]

This way I can split the project into multiple apps such as Customer, Computer,
Backups etc. and import the shared models from core. This allows me to split
the views and tests over multiple apps making the whole thing a bit easier to
understand. See the Notability note for more information.
https://github.com/netbox-community/netbox/tree/develop/netbox might provide an
example When doing the refactor I should correct the imports. The current
system is very annoying when I add a new object/class.

** DONE Hardware Model
CLOSED: [2020-02-14 Fri 20:28]

I'm currently unsure if I should implement a hardware model. With this model I
could add the hardware model to a device. Currently this capability is missing.

** DONE add a list of assigned users and computers to the license view
CLOSED: [2020-02-15 Sat 18:53]
** DONE Server mit NGINX aufsetzen
CLOSED: [2020-02-15 Sat 18:56]

- https://docs.djangoproject.com/en/2.2/howto/deployment/wsgi/uwsgi/
- https://uwsgi-docs.readthedocs.io/en/latest/tutorials/Django_and_nginx.html
- https://linuxconfig.org/how-to-host-django-with-nginx-on-ubuntu-18-04-bionic-beaver-linux

** DONE CustomerListView [3/3]
CLOSED: [2020-02-16 Sun 18:45]

add all the objects

- [X] Backup
- [X] Software
- [X] Users

** DONE implement permission decorators currently all the items can get viewed.
CLOSED: [2020-02-16 Sun 18:52]

I either have to implement a decorator for each object type or find a general
way. This problem is only related to detail views. The tables and lists have a
general way to check the permission. I maybe could get the model name from the
url, this Stackoverflow post might help:
- https://stackoverflow.com/questions/58307055/access-django-model-name-from-admin-url-pattern
and get the object with this function:
- https://stackoverflow.com/questions/28533174/programatically-accessing-django-models-from-another-app

** DONE Add tests for multiple nets and devices
CLOSED: [2020-02-16 Sun 18:52]
** DONE rename variables for the querysets to XXXRelations
CLOSED: [2020-02-16 Sun 18:53]

** DONE limit the queryset in the customer_table
CLOSED: [2020-02-16 Sun 19:13]

The queryset should only contain results which a users is allowed to see.

** DONE make sure the licenses models are correct.
CLOSED: [2020-02-16 Sun 19:31]

I think manytomany might not be the correct relation since a user should only
be attached once to a user license and a computer should only be attached once
to a computer license. However a user can stil have many licenses and a license
can still have many users.

** DONE tables problem
CLOSED: [2020-04-20 Mo 12:49]

This isn't fully working yet in django_tables2
https://github.com/jieter/django-tables2/issues/721

#+begin_src diff
-from django_tables2.utils import A
+

 class CustomersTable(tables.Table):
-    name = tables.LinkColumn('customer', args=[A('pk')])
-    nets = tables.LinkColumn('nets', text='Nets', args=[A('pk')])
-    computers = tables.LinkColumn('computers', text='Computers', args=[A('pk')])
-    devices = tables.LinkColumn('devices', text='Devices', args=[A('pk')])
-    backups = tables.LinkColumn('backups', text='Backups', args=[A('pk')])
+    name = tables.Column(linkify=("customer", [tables.A("pk")]))
+    nets = tables.Column(verbose_name="Nets",
+                         linkify=("nets", [tables.A("pk")]))
+    computers = tables.Column(verbose_name="Computers",
+                              linkify=("computers", [tables.A("pk")]))
+    devices = tables.Column(verbose_name="Devices",
+                            linkify=("devices", [tables.A("pk")]))
+    backups = tables.Column(verbose_name="Backups",
+                            linkify=dict(viewname="backups", args=[tables.A("pk")]))
#+end_src

** DONE implement a warranty overview
CLOSED: [2020-04-20 Mo 13:31]

2020-04-20
This is implementend now. I've setup the view so that customers can use the
view as well and the content gets limited to what they are allowed to see.

This view would show all devices which are running out of warranty, maybe this
could be shown as well in the CustomerDetailView. So that we would've a list
for the customers to see and one large list which shows the warranties for all
customers for internal usuage.

** DONE Fix the warranty in the device view
CLOSED: [2020-04-27 Mo 21:31]

With currently the containers don't disappear fully. Should be easy to fix.

** CANCELLED Move the lists to their own page
CLOSED: [2020-04-27 Mo 22:15]

Since I have more devices than I thought it would provide a better overview
than one big list. Forgot again what this exactly means.


** DONE [#A] cpu reduce required fields                              :model:
CLOSED: [2020-04-30 Do 14:54]

The CPU has many required fields but sometimes they don't make sense. For
example when we have a virtual CPU we usually don't need to know the specific
frequenzy.

** DONE [#A] the computer is missing a GPU                           :model:
CLOSED: [2020-04-30 Do 15:45]

CAD computers often have sppecial graphics cards which we should be able to
track.

** DONE [#A] Days need to be manytomany                              :model:
CLOSED: [2020-04-30 Do 19:21]

A backup can be run on multiple days

** DONE [#A] the IP needs to be able to be null                      :model:
CLOSED: [2020-04-30 Do 20:33]

Currently it's always required however when a device is in DHCP mode we can't
know the IP for sure.

** DONE [#B] Add the warranty to the Device, ConnectedDevice and Computer admin pages
CLOSED: [2020-05-03 So 19:32]

This is a relationship which a technician should be able to add directly on the
device and not have to navigate to the warranty option first.
** DONE [#A] check if the connected device and device model could get merged :model:
CLOSED: [2020-05-03 So 22:19]

When using the forms for warranties and net relations they both point to
different models but use the same logic so they always go back to the device
details.

However it's nice to differentiate between the connected device and the device
in the forms. So that one only can select the connected devices in the
dropdown. However the line between a connected device and a device is very thin.

** DONE Device Forms [2/2]
CLOSED: [2020-05-03 So 22:20]
- [X] Add
- [X] Update
- [X] Delete

*** DONE Warranty Forms
CLOSED: [2020-05-03 So 22:19]
- [X] Add
- [X] Update
- [X] Delete

*** DONE DeviceInNet Forms
CLOSED: [2020-05-03 So 22:19]
- [X] Add
- [X] Update
- [X] Delete

** DONE [#A] add a button to add a relation
CLOSED: [2020-07-05 So 13:05]

For the many to many relationships it might be a better idea to add them in a
separate form. So that the view displays the information but when you want to
add a new CPU for example it shows a button "Add CPU" which you can click an it
brings you to a new page where the PC is already preselected and the CPU can
get selected from a drop down.

The update form currently adds a new relation every time I update. The idea
above would be an easy solution. Another option to try out might be the
MultipleChoice field[fn:8].
Maybe with this I can limit the number of times a relation can be added.

** DONE [#A] Fix the table for the connected devices
CLOSED: [2020-07-05 So 13:36]

Currently there is a column for "Device_ptr" I have no idea where that came
from.

** DONE [#A] Add a view for DeviceManufacturer details
CLOSED: [2020-07-05 So 13:39]

since the DeviceManufacturer model now contains various contact details we
should provide a details view to let uses access those details.

** DONE [#B] Show all models in the admin interface
CLOSED: [2020-07-05 So 13:49]
:LOGBOOK:
- State "WAITING"    from "NEXT"       [2020-07-05 So 13:49]
:END:

Currently I'm hiding a lot of models from the admin interace to make it look a
bit nicer. However for the production site we want to work mostly on the
frontend. The admin page should then become really what it is. An admin
inteface so that we can delete or add models which currently don't have a
frontend interface.

** DONE [#C] Extend the CSS
CLOSED: [2020-07-05 So 13:50]

- A more centered layout would be nice
- Maybe some colours

** CANCELLED [#C] update the url code [fn:5]
CLOSED: [2020-07-05 So 13:56]

I'm currently not sure what I wanted to do with this.

** CANCELLED [#C] Add a check to see if a software has a license attached to it.
CLOSED: [2020-07-05 So 14:04]

I forgot the reason why I need this. Maybe to show in general if a software has
any licenses attached to it.
Add a check to see if a software has a license attached to it.If so it
increases the used licenses counter. Maybe with this this stackoverflow post
can help [fn:7].

** CANCELLED [#C] Inline Formfields like in the admin interface
CLOSED: [2020-07-05 So 14:06]

If already found various Github projects which might serves as an
example[fn:10] [fn:11]

currently not needed anymore I solve it with multiple form views because the
manytomany always generate a new object when updating.